Jim Manico is full of opinions. The founder of Manicode Security has advice on how to use the OWASP Top 10, on secure coding and especially on the OWASP Application Security Verification Standard (ASVS). He has advice for people starting out in security and all around thoughts on what it means to be a decent person. Jim is definitely one of those! He's also an educator, author, investor and entrepreneur. There are so many reasons to listen to this episode. Here are just a few: * Hear from one of the leading educators focused on helping developers code securely. * Learn more about all the important projects and initiatives happening at OWASP. * Get Jim's perspective on how organizations can best implement DevSecOps.
Jim Manico is full of opinions. The founder of Manicode Security has advice on how to use the OWASP Top 10, on secure coding and especially on the OWASP Application Security Verification Standard (ASVS). He has advice for people starting out in security and all around thoughts on what it means to be a decent person. Jim is definitely one of those! He's also an educator, author, investor and entrepreneur. There are so many reasons to listen to this episode. Here are just a few:
* Hear from one of the leading educators focused on helping developers code securely.
* Learn more about all the important projects and initiatives happening at OWASP.
* Get Jim's perspective on how organizations can best implement DevSecOps.
Key quotes:
* "Honestly, you shouldn't be basing a security program on the OWASP Top 10. The Top 10 is meant for one purpose only: awareness. This is not just my opinion. This is actually codified in the introduction of the Top 10."
* "Being a decent human being, being a community supporter, trying to help people out, giving free talks: you can call it being a decent person, but it's also a good life and business strategy."
* "Learn how to f-ing code. And you don't have to be an expert at it. You don't have to be a software engineer, but if you're an IT professional and you don't even understand the basics of coding, it's going to limit your capability because the best pentesters I know write scripts."
Related links:
* https://owasp.org/www-project-top-ten/
* https://owasp.org/www-project-application-security-verification-standard/
* https://www.synack.com/
Bella DeShantz-Cook: [00:00:00] Well, hello, Jim. Uh, it's so good to meet you. We're really excited to chat with you today. Uh, how are you doing today?
Jim Manico: I'm doing fantastic. It's my honor to be on your podcast. I love talking security. Um, thank you for having me here, Bella.
Bella DeShantz-Cook: Awesome. We're also joined by Jeremiah, my cohost and Jeremiah. How are you doing today?
Jeremiah Roe: doing get things fell. I am currently on vacation traveling. Jim, thank you so much for joining again. It's our pleasure to have you on the show. You've had such an impact in the industry and we're just happy to be able to highlight that. So thanks for joining
Jim Manico: I guess my marketing is working then. So, no it's thank you for your kind words. It means a lot to me. I appreciate.
Jeremiah Roe: So I'm getting started, uh, ultimately, you know, we just kinda, um, given your history and everything you've done in security. So. I'm kind of curious as to why you, why you decided to start mannequin security and why you focused your career around the secure coding aspect.
Jim Manico: This is my wheelhouse, right? I've been Mo I'm [00:01:00] primarily a developer. Even today. I don't think of myself as a security professional. I'm a developer and I, and throughout the course of being an architect and parts of my career, I had to learn about. How to defend my code, how to defend my applications. I'm talking like mid nineties when it wasn't really invoked to do so.
And I was getting attacked and learning about this. So I ran into Austin and my research, and I began arguing with Jeff Williams about security. And he was, he was very helpful early on. And I, as I continued to study of secure coding, I realized. Not much out there on this topic. So I began to collect the information that I was, that I was studying and starting to organize it for my own use.
And I, and like you, I started a podcast on application security because as I'm collecting this research information for my own work, I'm meeting all these rock stars in application security at the time of this really small, you know, unique community. So I did my own podcast to record the beginning of this industry and all the brainiacs who were.
Who are teaching me around about [00:02:00] secure software. I met Eric Sheridan, Dave Wickers, Mike Burstein, and all these original, some of the Andrew Vander stock and a lot of the early folks at old wasp. And so I eventually collected information and wrote a book. I wrote a book for Oracle on how to build a web application security.
And the Java language. And I was kind of bit from there. I just it's like writing a book was hard. It was not a profitable endeavor at all. It was a intellectual endeavor and a mental in a, in a, in a marketing piece, but also a way to organize my thoughts and writing that book is really what prepped me to be a teacher.
And so since I've been teaching for about a decade now, Monaco has been around for about six years and. It it's, you know, it's not just a business. It's my joy. It's I love teaching, you know, my grandfather and father are
Jeremiah Roe: I feel that for sure.
Jim Manico: and it's, this is something I love doing. So that's the kind of progression from developer to getting attached, to being introduced, to Owasso, [00:03:00] to writing a book, to starting a training company.
I feel very fortunate and I love what I do.
Jeremiah Roe: and that book was called ironclad Java. Is that, is that right?
Jim Manico: It's ironclad, Java building, secure Java web applications.
Bella DeShantz-Cook: So I know Monaco is kind of focused around teaching secure coding. Uh, can you tell us exactly what you mean by that phrase, secure coding and specifically, like, what is the difference between learning secure coding practices and just learning to code generally?
Jim Manico: like, for example, when I'm learning to code, I might be working with SQL SQL data access metalanguage right. The SQL standard query language. And if. Adding variables to my developer built queries with the raw string building. Guess what? My functionality works. And the application works and I can accomplish my business goals.
But one of the, one of the non-functional characteristics of [00:04:00] that way of building a query is that it leads to sequel injection secure coding is when you're just not satisfied just to build functionality that works, that you want to use something called a parameterized query and other techniques to access that data.
In a secure fashion, for example, another other example of secure coding is I'm using JSON web tokens, and maybe I'm doing a really bad job of verifying that signature, but my authentication system works. And then I find a flaw, like maybe I'm using an old library and I'm using, uh, I, the attacker can do a weird trick.
They can set the signature type to none to bypass my signature. Check on your JSON, web token. That's the insecurity doing it securely means I'm aware of this problem. I keep my library. And I have my server verify a limited number of valid signatures for that kind of Jason token mechanism. That's a little bit of extra effort and it's that it's not that [00:05:00] much effort, but it does require specialized knowledge.
And Bella it's a lot of minutia and this, I don't think people talk about this a lot, but doing defensive secure coding is like hundreds of requirements. The developer developer needs to get, right? Some, some are functions and some are non nonfunctional characteristics of software. So this body of knowledge is brutally complex.
There's a lot going on. That's why something like the ASVs that's when the old loss documentation projects, the application security verification standard, it's the combined wisdom of over a hundred professionals that come together to build a requirement list as a standard that the whole world can benefit.
Right. I'm an expert in this area, supposedly. Right. But as I work on the standard, I learned, I learned more than I give. And I learned a lot just by keeping a close eye on the standard. Cause I guess the collective wisdom of security requirements for many, and this is the minutia [00:06:00] of secure coding, you know, written down and get up for us for our salts to benefit.
Jeremiah Roe: So I'm a, I'm a huge believer in contributing into the community, not just utilizing the resources that are there, but also giving back. And to that point, I'm kind of curious, you know, how did you transition from. Chatting with folks around secure coding and, and, and having those, those community conversations to becoming a really, a really big contributor in something like, you know, oh, wasp or, uh, what a wasp is doing with the ASB.
Jim Manico: Yeah, that's an interesting question. I mean, this is just my nature. It's like, you know, I don't have kids. I have free time. I've run my own business. I don't have like the standard nine to five job. I have a lot of freedom in my life to do what I want to do. And it's very selfish. Of me to do this because the more that I give to Owasso, the more I do freak talks, the [00:07:00] more when some random person says, can you please mentor me?
I need help. The more I do that, what's the side effect of that kind of behavior. It's, you know, I, I, I get recognition. I get kudos. I get the guy who I helped 10 years ago. Some like low level engineer who like, could barely talk security. And I gave him my time. Guess what? He's now. Guess what he's doing now?
Eight years later. Oh, he's a developer man. Hiring me at no small expense to do like professional instruction with them. So being a decent human being, being a community supporter, uh, trying to help people out give free talks. It's just, it's uh, it's it's you can call it being a decent person, but it's also a good life and business strategy.
I don't do sales. I don't do more. I do sales. I'm sorry, I don't do marketing. I just try to like give to the community. And as a result of that, people want to hire me to do training. And so. It's my way of life and it's, and I, I think it's an important one and I'm not perfect. I make mistakes, but [00:08:00] I feel the more, the more I give out to the community, the more I benefit from it.
And the more my business benefits from it and more emotionally benefit from it. Like, you know, what do I get from a loss? People saying, thank you to me a lot are grateful for my effort and that at warms my heart, right? So this is my business and life strategy to give as much as I can to the community.
It's something I've done for a long time. And I'm very selfish as I do it.
Jeremiah Roe: I really
Bella DeShantz-Cook: if you could describe that as selfish at all, but completely fair. Understood. Um, so alas, top 10 is widely used in the industry. Uh, I've referenced. All the time in, in my career. Uh, how does the top 10 and other OAS projects as well, including ASVs that you've mentioned. And I know there are plenty of others.
Um, how did these projects help organizations learn about security and kind of focus in on their own security posture?
Jim Manico: Certainly, you know, like why, why did we get a lawyer? Why [00:09:00] do we hire a lawyer? I mean, the laws are written down, not just go read the law bureau lawyer because the interpretation of the law is super complicated and I, and the, in the laws. So it, I kind of see like security in that vein as well. Right. And so. Um, I'm not sure. Can you give me your question one more time? I want to make sure I'm going down the right or the right path. I I'm going somewhere, but give me your question one more time, Bella.
Bella DeShantz-Cook: No, you're good. I think what I want to talk about with you is how a wasp. I mentioned top 10, because I think it's referenced and used a lot, but really every single OSTP project, how do, how do these projects help organizations learn about security and focus? Like focus on improving their own security.
Jim Manico: and I want to strongly differentiate cause the purpose of the top 10 and the purpose of the application security verification standard ASVs are very different. The point I was trying to make is that I get, I get it now is that, look, we go hire a lawyer because we have massive [00:10:00] complexity. And when you're a developer who hasn't really addressed security in your career, much suddenly learning about security is like interpreting the law is WIC.
There's so many little details you have to get. Right. And it's esoteric knowledge less. So these days, but it's esoteric knowledge. So what does OSTP top 10 do? Oh, us top 10 is really. Someone who is new at web security and API security. There's also the M really the ASVs is really the w ASVs web application.
Verification. There's also the mobile and IOT and IOT and other ASVs. But back to the last top 10 it's for initial awareness, if you're a developer or a security professional, and you want to learn about web security, this is a good document to start your research and web security on. But the thing is you should read the top.
And then put it away because honestly you shouldn't be basing a security program or anything on the top 10 dos, top 10 is meant for one purpose [00:11:00] only awareness. This is not just my opinion. This is actually codified in the introductory of the top 10. That explains, please don't use it as a standard because it's just a top 10 list, but it is a defacto standard because of how.
It's an easy thing to attach to. Oh, my product supports the O S. Who cares? I mean, like it's nice. I mean, I don't want to be that, that much a curmudgeon, but to some degree it's an incomplete message. Cause I'd rather use, start addressing, Hey, I, I, my testing tool handles ASVs level one requirements, according to the 4.03 standard.
Oh, that's a bit more comprehensive. So do you watch top 10? And I love it. I'm out there giving OWAS top 10 talks like three times a day. I went to LinkedIn and said, Hey, anybody. Oh, I'll stop 10 talk. I got my new talk and I'm like, okay, I'm booked now. Whoa. So that I love the OSTP top 10 for breeding awareness for giving conference talks.
It's one of my training modules. Yeah, Nan, [00:12:00] stop. Put it down and pick up the a S V S the O us application security verification standard, a comprehensive, almost 300 requirement standard to help you as a web developer or a test. Evaluate web security of web software, and now he can base a program off the ASVs standard.
We can fork it for our own standard and it's, I love the team that works on it because it's like Daniel Cuthbert, Andrew Vander stocks been a big part for awhile. We have ELR Lang who is a new leader of the project. Uh, Joshua Grossman. We have, uh, smore and several others who participate are just amazing.
And here's how I see the group. We are the biggest group of pedantic bastards I've ever worked with. Oh my God. We will like fight to the death on like three words in a requirement and get
Bella DeShantz-Cook: I don't think I've
Jim Manico: off at each other because they want those out there. You are. You're repeating the word [00:13:00] cookie twice in that sentence and it doesn't make sense, but I love it cause we all forgive each other and we move on, but we've and this, we need this.
Jeremiah Roe: change the context of this thing or.
Jim Manico: We need that level of meticulous eyes on the standard though. And I want to applaud the here's the real driver right now. I want to give him credit for what he's been doing. The person who's really been driving the most meticulous work of the standard and pushing us towards being a much more respectable standard is a gentle pro a gentleman from Estonia named, uh, ELR Lang and his work was so great.
He was pestering me so much. I'm like, I am so sick of your constant pestering and demanding me to do more work. Guess what? You're a project leader now. Boom. And he I'm being a little joking here, but he's been a great project. And he, he, he, he has that like real meticulous eye on each requirement and the structure of our document and everything that, he's one of the reasons we just released 4.03, which really [00:14:00] was a really great job.
Thank you. We LAR and thank you, Joshua and Daniel and Andrew and everyone else who's volunteered their time to make the standard of diesel.
Jeremiah Roe: So I've, I've got a question around this. So looking at a wasp top 10 more as treating sort of symptoms instead of addressing sort of underlying root cause, which is what I think of when I think of ASVs right. So building that foundation and treating a root cause of insecure coding practices to both educate and to inform those developers, I'm kind of curious, right?
Because there's, there's been a new inclusion in just the toe. Oh, top 10. That includes insecure. And so as, as a developer, right. I referenced the top 10 and I look at these things so that I can. Being formed as I'm coding things, following the ASVs, but there's that interesting addition to include insecure design.
So, so, so when, when we look at that, how does that tie back into the [00:15:00] ASVs and from a testing perspective, just, just for those testers out there, right? How do they, how do they begin to test for insecure design? Do they, do they reference back to the ASVs? How does that tie.
Jim Manico: Let's take a step back first. Let's let's look at how the OSTP top 10 has changed in 2021. If you look at the 2017 category for, for data loss, we called it like a. I forgot what I want. Let me go through this stuff real quick. Let's go. We gotta go check this out. So I'm going to open up my slide deck here.
I'm I'm so ready to talk about this. So. Um, I want to look at how the cryptic graphic category changed from 2017 to 2021. So I'm scrolling. I got my slides up. That's my whole world. I'm ashamed to say I'm a PowerPoint expert. So cryptographic failures is the current category this year. That's the root cause.
Back in 2017, we caused it. We called it sensitive data exposure. That's a broad [00:16:00] symptom, but the root cause. You messed up crypto, right? Use the bad library or the wrong algorithm, or you're not doing proper key management. So I like that change. Cryptographic failure is more like direct to that developer.
Let's get crypto. And now we can point in that like a TLS, right? Dukey management use a secret fault, have a proper key lifecycle user, right. Algorithm use a well-vetted library and more, we'll take a look at the cryptographic storage, cheat sheet to see some of those details. And so now let's look at secure design.
This is a little controversial, but I like it. What we're trying to say, what, what is this, what is this new requirement really trying to say? And this is the fourth one. Let me read this from the, from the document. Yeah. um, insecure design is a new category for 2021. With the focus on risks related to design flaws.
If we genuinely wish to move left as an industry, we need to do more threat modeling, secure design patterns and principles and reference architectures. So it's not just threat modeling. I [00:17:00] like the term reference architecture because if you're a threat modeler and you're just walking into a meeting, start talking about secure.
You're like in one of those dinosaur blow up costumes, you ever see those old dinosaur blow costumes like this you're flopping around. Like, so when you go into a threat modeling session, you want to have a plan reference architectures that your company approved, secure design principles and standards.
You're trying to conform developers. Don't flop around, have a plan to conform the different projects to your standard architectures. And then I think dessert, secure design is really useful, but the point is that we're not just the problems are not just individual technical vulnerabilities anymore. As we move to these really complicated end tiered microservice-based systems to software is now how we design that system is really critical.
Go back to your traditional web app in the, in the late nineties and two thousands, it was a web server and a database, and it was really straightforward architecture. [00:18:00] Most of the time today I'm in the cloud. I got the Hadoops and JWT verification, key management servers, and I got I'm breaking things up in a hundreds of microservices.
I got to have re vocation end points, and my it's just, there's a lot more
Jeremiah Roe: party libraries. Yeah.
Jim Manico: I'm using tons of third-party libraries because of the massive complexity, modern software, sometimes as a part of planning that dive at the architectural level, I think is really important. That's what insecure design is all about.
It's acknowledging that problems are more than just low level technical vulnerabilities. And if you don't think about your zine ahead of time, trying to unwind and insecure design is massively expensive. So let's do some threat modeling when we're changing our architecture. Boom
Bella DeShantz-Cook: It sort of sounds like the 20, 21 top 10 is very like in keeping with the whole trend of shift left. Like those multiple changes that you mentioned really. [00:19:00] Make it seem like, I know earlier you talked about how, oh, I was top 10, should not be used as a testing standard. Um, and I think in my experience, I see that, like, I see a lot of customers who say, Hey, can you test for us top 10?
Um, and I think that these changes really helped to kind of say what you've been saying, which is like, this should not be a testing standard. You have to do more, you know, like this is about kind of beyond that. If that makes sense.
Jim Manico: the way I dress this as an educator, as a, as a tester, when somebody approaches you for the top 10, there's nothing wrong with that. That means what they're really saying. I need help with web security issues. You've got transit and then know, you know what? I can definitely help you with that OSTP, top 10 testing and here's, and here's the category.
Here's all the things we're going to be searching for. If you're a good testing company, you have a clear list of like what you're going after. Not just here's here's what you don't tell customers. Yeah. Just trust us. We're going to flop around and do some testing. We'll find some stuff. No, no. You usually want to let customers know what you're up to.
Transparency [00:20:00] helps. And I just, I don't and people asked me, I need a boss top 10 training. So I have my, like my content broken down into individual learning modules. I'm like, yeah, I can do that. And I give them a list. Oh, was top 10 is one of those items. And I showed them all the 40 other modules I offer.
And it's never a problem. As long as you convert the top 10 request from a customer into just a general web security. I've never had a problem with that conversation. And in fact, it, it gives, it's actually a good thing because when they ask the question, I know exactly when they asked the question, I know where they're at and what they're asking for instantly, and I can give, I can help them out.
I know I can lend service to them.
Bella DeShantz-Cook: Yeah. So I want to talk a little bit more about O S ASVs. Um, so I'm a huge fan of ASVs. I used it in my previous role as a pen tester. Uh, I talk about it all the time. Now my coworkers probably get really sick of hearing about ASVs from me. Um, but one of the things that I talk about a lot [00:21:00] with it, which you mentioned already, uh, we, we mentioned already was that it's a different approach to security than a lot of other, um, References out there.
And then a lot is different than a lot of bug bounty and pen testing can be it's that kind of a proactive looking for security controls instead of finding exploitable vulnerabilities kind of mindset. Um, so why is this approach to security important?
Jim Manico: You know, uh, ASVs is supposedly a set of requirements that both a tester and a developer can benefit. And so let me, let me go. I'm going to open up ASV, let me open up a S V S while we're talking and giving an example of some of these requirements for the sake of our conversation. So I'm going to, I'm going to, I'm going to go do a little Google on Olaf.
And get hub because all of our work is done. Very transparency transparently right now. [00:22:00] So I'm at the main get hub repo now, and I'm gonna click on the Five-O directory, English, where our current work pushing towards the five O standard has done. That's where you see the most recent changes. So I'm going to look at, I'm going to go a little deeper into validation and sanitation and in coding section here's some requirements verify that the application has defenses against HTTP parameter pollution attacks, particularly.
If the framework makes no distinction about the sorts of requests. Get posts, cookies, headers, environmental variables is so I don't deal with this much in my practice, but that's the level of detail that you'll see in a requirement. And this is, and this doesn't give the exact answer. We got to do a little more research beyond the requirement.
Well, how do I test for HDTV parameter pollution attacks? There's a whole body of knowledge around that as a developer. How do I stop it? Sometimes my framework handles it. That's mostly the common answer. This could be some other, other problems as well. Uh, other things I need to do at my framework level to stop this.
So [00:23:00] the requirement is not about being, giving you all the answers it's pointing to the problem, as clearly as we can. Now, some requirements do get a lot deeper. How about this? Verify 8 5 1 3 in the Five-O bleeding edge. Verify that all input HTML form fields, rest requests, URL parameters, HTP, headers cookies, batch files, RSS feeds is validated using positive validation and allow lists.
So we want to verify that all input, all of it, entering your software, goes through some validation. We're not specifying what the validation is. Am I going to do a regular expression? Am I going to do numeric validation? Am I going to stand up? Am I going to do it, use an HTML, validator or sanitizer to do my validation step?
That body of knowledge is super complex, but we're pointing a, every input should have some kind of input validation, and we try to get deeper into that into the different, different sections. But it's only about just the section input validation and. It's about like [00:24:00] 20 or 30 requirements. It's not all the answers.
It's not like a it's as comprehensive as we can get without starting to branch into details that are not common to all of us building web apps. When I start talking about there's this problem in react or angular, and you have to do this, we keep away from those requirements. We keep it. So it's general points you to the problem from both a defense and attack point of.
To to understand what a complete assessment or secure coding program would start to look like. So I'm not sure if I even answered your question, but.
Bella DeShantz-Cook: No, I think that, that, that absolutely hit it. I think, you know, again, I said this already huge fan of ASVs and I think there is so much value in that type of approach. Um, and I think it's important to kind of discuss why, like, where does that value come from and how that is used both by, like you said, both by testers and by developers.
Um, so it's interesting to discuss that.
Jim Manico: I mean, one [00:25:00] more brief note to that. The way I think of it is like, as a developer, I'm from the late nineties, we used to did a lot of waterfall stuff. So we really planned out our requirements before we built apps. I see less and less of that in the modern era. We were very requirement driven. I started my career at GE.
Very engineering, heavy firm is actually a big company back in the nineties. And so we were engineers and we wrote out our, our requirements. So to your answer, my answer to your question is this is the list of requirements that's in the language. Mostly for me in developer land. This is how I built, like, cause I'm sorry.
I rarely see testers being like let's ever requirements doc, to plan this pen test, you know, I don't see, I, I see a lot. I don't want to pay, I got my tools. I've got my I'm gonna go. Do some, do some scanning and hacking pentesters, but for developers, we're a bit more requirements driven. Here are the specific things that we need to build in this project for it to be complete our list of requirements, business requirements, as well as [00:26:00] technical requirements that, that drive. And so this re, and this is in the language of developer building requirements that attackers and testers should also benefit from. And to your point, Bella, that we've moved, we've moved in that direction from the original version, which was mostly testers who built it. Now we're developer pack who are mostly working on.
Bella DeShantz-Cook: I think so. I mentioned already, I, I worked previously as a, as a penetration tester, and this is ASVs was a resource that I used a lot when communicating with customers. And I think like one of the values that, that I saw and continue to see is that a question that I have heard from customers when doing pen tests without this type of resource is. Okay, but what was good or, okay. Ish or w what, what, what was going on other than, oh, no, this is a critical vulnerability. And I think, uh, to your point, there are, I think when developers are looking at an application and thinking like, you know, here [00:27:00] are all the things that we have in place, and then they get a pending.
Result that is here's one critical vulnerability in only one area. It's, there's so much information that isn't there. And I think using a framework that, that kind of covers everything is really helpful when you're able to communicate like, yeah, you know, we are, we're only delivering this one vulnerability, but here's it's because here are all the other things that we looked at and didn't find issues.
Jim Manico: the other really great point. It's it's, it's a way to establish a baseline between a customer between. Uh, tester and the recipient of those results to understand how complete the test was. It's a great way to measure if someone is doing manual pen testing for you to eat, or if they have a tool scanning you even to measure how complete of a tool that really is, is to, is to compare it against the ASVs for complete.
So again, it's got multiple uses for, and I was even told that like, there's the Owasso testing guide, which is [00:28:00] amazing resource. I know some testers who don't use that and use the ASVs instead because it's, it's, it's, it's not as detailed per requirement as the testing guide, but it's more detailed in the completeness and the number of requirements to give teachers an idea of, of things they should look for and to give customers and understand that.
Uh, what the testers actually look for and a good tester will not only point out what was wrong, but will point out the good things they run into. Hey, I tested for deserialization like crazy. Didn't find any I tested for this. Didn't find any, and that's way more of a mature pen test report out than if you're just showing them.
If you crits.
Bella DeShantz-Cook: I think additionally to something. And again, this is me as a user and lover of ASVs, uh, something that I've noticed is that like ASVs offers the language to say, you know, here's an issue that we found and here's the way that it should be. Um, so it's not the. The requirements [00:29:00] aren't, don't have this vulnerability.
They're more like have this protection in place to prevent against this vulnerability. And so something that I've noticed in my time around pen testing is that when you're delivering a vulnerability report and all you say is, oh no, you have this issue. Uh, it's really hard for a developer to like, do something with that.
Uh, unless they're super well versed insecurity as well, because. That's not like a developers, aren't coding in issues intentionally. They're trying to code things that work and that are secure. And so with ASVs having those, like, here's how it should be. Uh, that language is so useful in bridging that gap that we've been talking about.
Because as a tester, you can say, Hey developer, like here's what you should do.
Jim Manico: I'm I'm with you, like look at the bleeding edge requirement, 5, 3, 4, verify that data selection or database queries use parameterized queries. The main defense stop sequel injection. I did not help you as a tester find sequel injection. I, I TA I mentioned that that you should be testing for this. [00:30:00] And look for this, and this is a white box test.
This is actually looking in code to see how the coding construct was done. That's not a pen test requirement. That is a, is a code review requirement to look for. So that shows developers how to really check, to make sure the developer built back control correctly. And it directly informs a developer.
What the control. This is another reason why I feel like ASVs is migrated more into a developer centric standard, right? Because 5, 3, 4 is not a pen test requirement. That's a code review or developer requirement. And I think it's useful. I think it's the right way to do, to do thorough test.
Jeremiah Roe: I like to think of sort of pen tests as, as an obvious result as to how something's been. Right. And so if you've put something together and maybe, maybe you weren't aware of something like the ASVs, or maybe you weren't aware of a certain processes that you need to fall, uh, follow, like, like [00:35:00] parameterization or, or, or filtering or something like that.
Right. Then, then, uh, a pen test or. Or something along those lines are directly going to call out those aspects in which you failed to implement. And then you can reference back to something like the ASVs to incorporate a thought pattern while developing that, that process flow. Um, and, and, and so to that, there's, I, I understand there's, there's different levels with, with ASVs in the framework that's that's around there.
Can, can you talk a little bit more as to why those different levels are.
Jim Manico: Sure right now, there's level one, two and three in level one is basically the way it stands today. This is what every app should have for level one. And these are things that are very easily testable. two, we have this, this apps, what's more sensitive data, a bit more critical and you know, level two are, is lot less can be tested with automation.
Level three is for like critical infrastructure [00:36:00] requires going to require more manual testing. And I dislike these categories is one of my pet peeves. I'm trying to. Migrate ASVs away from just being a testing standard for levels and do what all other standards do, make them risk levels based on the, based on, uh, like I want to map this more towards what NIST special publication, 863 3 is doing.
That's the digital media standard for U S government digital identity standard framework for us. And they're AAL level one is Republic data only. And AAL level two is for sensitive data and more, more serious risk. And AAL level three is like Fido and smart card for more infrastructure level security.
These are risk levels. And the fact that we made it testing level, again, this started as a test pure stat testing standard. I strongly disagree. With, with these three levels mapping to different testing techniques. Cause, cause cause good pen testers use some scanning tools and [00:37:00] I got to interpret scanning results with a pen tester and what's automated and what's not is really a muddy topic.
I don't know any pen tester who only does manual testing. They're all being aided by a wide assortment of automated tools to get your job done better and faster and more thorough. So, so let, let I, and we're in the middle of this discussion, right? In fact, if you go to the issue tracker, by the way, Bella, anyone who really loves ASVs, you know, what they do, they could tribute, um, luring you in to submit issues.
You see anything wrong? I want your feedback, but if you go take a look at, uh, let me, let me do a quick search. Based, let me see if I can. That's my, that's my big cry. Now let's make these levels risk-based instead of testing based, if you look in our issue tracker issue one, one to seven, you're going to see you LAR and myself, and some others debating whether or not we should be [00:38:00] testing level, which I don't agree with or brisk level with more in line with, with NIS standards and, and we're, and we're debating that right now.
Joining the debate again. Issue 1, 1, 2 7 at the AIS. Get up repo, jump in and give me your perspective. We want community feedback here.
Bella DeShantz-Cook: I think it's funny that we are talking about this, because one of the biggest questions that I've gotten from, from customers when I've talked about ASVs is like, if I say, okay, based on what, you know, the. What type of business you do, the things I'm going to test, we should probably test with, or like we're going to use ASVs level two.
Uh, and so many times I've gotten customers say like, well, isn't level three, the best I want to be the most secure.
Jim Manico: oh, oh, oh no. I'm going to help you out here. Bella. Bella though, work with me here. Hey Bella, I think we should do a level two, a level two ASVs test. What do you think of that? Bella? Go ahead.
Bella DeShantz-Cook: I want level three. It's the best. I want to
Jim Manico: Oh, I agree. Level three is the best Bella. Absolutely. We should do level three [00:39:00] and that will be an extra $35,000 please for that
Bella DeShantz-Cook: Yup.
Jim Manico: like where's the check box, a level one, 10 grand, a level 2 25, a level three. We're going to, oh, I, you know what I agree with you, you know what customer you are, right? You're brilliant. You've read the standard. Right. You know what to do? I'll take your extra 35,000 in cash or gold bullion or.
Jeremiah Roe: proactive. You've got it.
Jim Manico: And also bill I'm joking around, but that's a fair answer. If you want an AA level three assessment, you gotta pay, or someone's really nice giving you like weeks of free service, but a level three, you got to pay and there's nothing wrong with that security assessments, expensive and there's, and they were like lawyers, we're specialists.
And you need us right now. So a level three, isn't that cheap and that's okay.
Bella DeShantz-Cook: Yeah.
Jeremiah Roe: So, so shifting a little bit from ASVs, but still staying on the development topic. Right. So you've described yourself as sort of the self-professed like developer, both in [00:40:00] nature and heart, and you originated from the developer space. I'm kind of curious, um, for those individuals coming up into the development space, what, what is sort of that, that thing that you would recommend to them that for those individuals just getting started, right?
What would you recommend to those endeavors?
Jim Manico: At what point at what part of their career is like people who are just entering, just getting started with an it career. Look, every time I say this, I upset people I care about. So the, so I just want you to know when I state this opinion, I'm not trying to offend anyone. It's just my opinion to have a good career.
And the, that comment, the controversial comment is learn how to FN. And you don't have to be an expert at it. You don't have to be a software engineer, but if you're an it professional and you don't even understand the basics of coding, it's going to limit your capability because the best pen tatters, I know, Hey, they write scripts.
They automate some of their work. And you don't have to be like an Uber, [00:41:00] like super coder, who's coding 60 hours a week or whatever, or more, but have some basic understanding of what the discipline of software engineering is. Do some of it yourself and coach take one. There's like a million online free classes to learn the basics about JavaScript.
It would take you, even if you're a non coder an hour to watch one of those tutorials and write some basic JavaScript that runs in the browser. And guess what? Now you're a little bit about. So when I say this people who are in management or people who do GRC take great offense, but I, and I don't say this to offend anyone.
I'm just saying that when you have a base knowledge of doing some coding, it's going to benefit every other type of it discipline, and I encourage you to do it. Nobody wants to hear this Jeremiah. They get mad.
Jeremiah Roe: I actually align with. Okay.
Jim Manico: One of my good friends, Ben and GRC blocked me when I said this on Twitter.
And I'm still, I'm still hurt because he was because [00:42:00] he's a GRC guy who did not get a job he wanted because he didn't know how to code. And I wasn't saying it to hurt his feelings. His, I just, a lot of people don't code take. I can, I can get a hundred people throwing smack at me because I say this on Twitter.
Hey, start your career. Learn to code. Here's a little tutorial. It'll benefit. Your career is going. It's controversial for some reason, but I stand by it.
Jeremiah Roe: I think if you can code today, it's a little bit like being a wizard, right. And someone will tie this back to something. Um, if you can code, you can, you can almost create anything that you can think of. In addition, it's going to benefit anything that you do going forward, because you can both automate, you can streamline, you can alter the effectiveness.
There's so many things you can do from a coding perspective. So I align very much with that statement.
Jim Manico: yeah, that's the first thing I would do in my career. Learn a little bit. And here's the second thing I'll recommend, right? Here's number two. Understanding. How AWS works, understand some of the basics of what it means to [00:43:00] use a cloud service to Deloitte, to deploy software even a little bit. Well, even just like for me, I spent like an hour messing with some of the AWS consults and I'm like, boom, it opened my mind up so much to what capabilities are there.
So I can talk about it more. It took me an hour of looking at their admin screens to have my brain explode. So start to understand cloud deployment of software. And that's the future of the world and understand a little bit about coding and it's gonna help your career in really significant ways.
Bella DeShantz-Cook: So continuing with advice, but I want to hit kind of a different crowd here. Do you have any advice for organizations that are, uh, like beginning to implement a DevSecOps program?
Jim Manico: So that's how I'm like, wow. You just like threw it. That's not a softball. That's a loaded question. You just threw it at me. How do you start yet? Sec ops. Right? I'll give you some, the thing is I don't like talking about dev sec ops, cause it's [00:44:00] usually like a lot of buzz words and it's very like, non-scientific I talking about digital signature types for JW validation.
That's hard science. So dev ops. All right, I'll do it. I'll do it. It's about it. Dev ops is all about automation. Anyone tells you differently. They're just wrong. It's all about. Uh, maturing your automated capabilities and he asked there's a culture change. Where's my violin. Oh, dev ops culture change. Yeah, whatever automation, CIC D is what dev ops is all about.
And in the world of security, it's about automating your, your security testing program. So here's my advice. My advice is get, get an automated pipeline going. That does one thing for testing, just to start. Run static analysis against your code base as a blocker, I'm going to recommend a tool that I'm affiliated with.
So I'm a little biased. I'm going to recommend a tool called SEM grip from return to court because it's free. And I ran a 5 million line code base in like 50 seconds. Boom. Now I can integrate the, get up automated pipeline easy. So that's one, I'll just one thing to [00:45:00] get started wire in static analysis that runs fail.
And use it as a blocker before you allow a check in to come in. There's one achievable, simple security goal from a dev ops pipeline perspective. Here's the second thing I would do. Why get dependable up and running to give you PRS? But when your third-party libraries are out of date, there's another thing.
And those two things alone. If you can mature your capability at running static analysis and running software composition analysis, third-party large library scanning as a blocker for every check-in. Whoa. Now you're starting to do, I don't care about dev ops dev sec ops, right? The security testing part of dev.
That's a good place to get started. I would dare say I wouldn't start with dynamic scanning in the dev ops world. It takes too long to run a dynamic scan. I wouldn't use a traditional static analysis engine in a dev ops workflow. They take hours to run and devs don't want to wait hours to finish a check-in.
So use one [00:46:00] of the newer lean mean fast static analysis engines to scan your code for security and check your third party library. First for out-of-date security issues as your first main tasks to automate security and the dev ops pipeline. Boom. There you go.
Bella DeShantz-Cook: I like this advice, because I think like we're having a conversation. A lot of times I hear this a lot, at least, uh, security. Is is like an annoyance and a blocker and it gets in the way of things. It slows processes down. And I think sometimes there is, you know, I'll say I've experienced it. There's some animosity from developers when working with security professionals because we get in the way.
Um, and I think it's really cool to have this conversation where, you know, we've been talking about a standard that is like, kind of designed for developers to use. You're giving advice. That is, you know, you're addressing that knowledge. Slowing things down is not fun. Um, I just think it's really cool to be able to have this conversation while you know, being conscious of the things that people don't like [00:47:00] about security.
Jim Manico: absolutely. We w w one of the power of dev ops is to get security professionals out of the. It takes a huge amount of automation and scripting work and similar to get there. But the point of dev ops is, is I now I'm going to check in code and I want that code. That piece of code is, is I want to go to production.
So we're going to run through a bunch of static analysis testing pass. I'm going to look for third-party library, scanning pass. I'm now going to do the build and get ready to deploy. I do pre-build testing and those tests all pass. So guess what? You're like. And who stopped me? No one. How about this? Oh, can we talk about that requirement first?
No, you're out of the way. Or can we do a threat model first year out or I need to review those results. You're out. Any people who get in the way of a live release, that's what dev ops is about. And it takes mammoth automation, security testing, and, and discipline process to be able to get that reward for all of your work and building a [00:48:00] proper testing pipline.
Jeremiah Roe: so to those very points right there. How do you see the overall shift left, you know, mentality impacting the industry as a whole? Um, is it shaking it up a bit or, or what are you, what are you personally.
Jim Manico: it depends on, it depends on the kind of company and the pressure. They have to write secure software. But one of my, one of my customers doing training with me for seven years now and. There and a medical field and they're doing well and they, they really care about not hurting people. So their, one of their, one of the risks they worry about is like public safety and similar.
And so every, I got to watch what I say with them because they take all my advice and, and they, they, they take my advice seriously. So I'm trying to say, and I've watched him like seven years ago. SQL injection was an issue. I look at them today and they're running a pretty bad-ass scanning, uh, Uh, collection of tools and testers at their stuff.
And it's bouncing. It's not [00:49:00] because I'm not because I'm dumb a DJ it's cause they, they, they did the work for years and developer security was never questioned for them. They're in the world of public safety and medical and they just do it and they little by little but shared their capabilities over seven years.
And now they're security rock stars. Absolutely rockstars because of their subtle shift left over many years.
Bella DeShantz-Cook: thank you so much for putting up with all of our questions so far. Uh, is there anything else that we haven't covered that you'd like to share today?
Jim Manico: No, it just, it, again, I think it's a good life and business choice to volunteer your [00:52:00] time to help the OSTP foundation do better help everyone do better at web security. I'll be looking for your I'll be looking for your issues or PRS or any feedback you have your, if you love it, if you love ASVs as much as you do, then you will, you will share your amazing knowledge with us and help us out the project.
So I'm, I'm
Bella DeShantz-Cook: Now that I know now that I know that y'all are super pedantic, I will feel right at home there. You'll you'll see me. You'll see me on GitHub for sure. Um, again, thank you. So, so, so much for taking the time today,Jeremiah Roe: and lastly, what's one thing people wouldn't know about you by looking at your LinkedIn.
Jim Manico: Here's an honest truth, Jeremiah, and it's, it's not, it's not something it's just the way I'm built. I'm I'm pretty transparent open book. I don't, I don't, I don't carry a lot of secrets. I don't have a secret vault. I have a big mouth and a lot of like open nature and I. Even when I don't want to, I usually just say it like it is.
So I don't, I don't, I don't, I'm not built with a lot is I don't have a secret vault. I have a big mouth. So there there's some people everyone knows about me who knows me is I'm a big talker and I'll say anything and I'm an open book. All right. That's it. That's my
Jeremiah Roe: That's awesome. Thank [00:54:00] you again, like Bella said, thank you so much for joining the show. I know I've certainly enjoyed speaking with you today. It's been a pleasure.
Jim Manico: Jeremiah and Bella have a great day. Thank you. So.
Bella DeShantz-Cook: Yeah. Thank you so much, Jim. It was really great.