Using a static code analysis tool is a common — and sometimes dreaded — part of the development process. These days, there are a dizzying number of choices available, from free open-source tools to fancy commercial products, which means it can be confusing figuring out how to choose the right one for your development team.
We’ll get to that, but first, let’s talk about the different types of application security testing tools.
What is Static Code Analysis?
What is a static code analysis tool?
Static code analysis tools, also known as static application security testing (SAST) tools, have been around for many years. These tools are a type of software that scans an application’s source code and summarizes any security vulnerabilities before the application moves to the production environment. Over the years, other automated security testing products have emerged, including dynamic application security testing (DAST) and interactive application security testing (IAST).
According to Nicole Schwartz, a product manager specializing in application security testing at GitLab, a popular code collaboration and DevOps platform, using a mix of all these tools is often the best approach. While SAST tools scan through source code to locate vulnerabilities, DAST tools don’t bother with source code, instead running against live versions of applications.
“DAST is literally attacking a live website,” Schwartz said. “The nice thing about DAST is that it’s language agnostic. However, it does need a running environment to hit against.”
Because SAST tools scan through source code, the SAST tool must be compatible with, and written for, whatever programming language the development team used to write the application being scanned. An advantage of this approach is that SAST tools can pinpoint security vulnerabilities to specific lines of code, Schwartz said, which in turn can help speed up the process of troubleshooting and patching.
Though SAST and DAST tools have their differences, using them together ensures a more robustly secure application.
“SAST is for my coding mistakes, and I need to find those. DAST could be for deployment or implementation mistakes, and I need to find those,” she said. “If you implement all of these steps in your software process, you’re going to catch everything. As opposed to if you only concentrate on one — even if you do that one perfectly — there’s still going to be stuff that a SAST scanner can’t catch or a DAST scanner can’t catch.”
The Differences between open-source and commercial SAST tools
The way your development team is structured and works should be taken into account when looking at pricing for SAST tools.
“Some tools charge by lines of code,” Schwartz said. “Other times it’s how many developers you have.”
But companies don’t necessarily need to pay for SAST tools, especially at first.
“There are many free scanners,” Schwartz said. “Start with that and show the value of it to your management, and then maybe upsell to a commercial tool because that’s going to be more usable.”
She said that, while many open-source SAST tools do a good job of catching vulnerabilities, they fall short when it comes to support and user experience.
“There are many free scanners. Start with that and show the value of it to your management.”
A paid-for scanning tool is “probably just going to be a more polished product,” Schwartz said.
“Some of the more advanced tools that have support, they gather a bunch of information about your environment,” she said, which allows them to cut down on the number of false positives reported based on whether it’s relevant. “Or maybe this dependency has a flaw with this function. It will actually check all your code to see, did you ever call that function? If you didn’t, then don’t worry about updating that dependency — whereas the open-source tools are not going to do that.”
Some tools can track how parts of your codebase interact
ShiftLeft, a company whose main product is a SAST tool, falls into this category. ShiftLeft takes its name from the idea of moving security considerations earlier in the development lifecycle, or “shifting left” in security-speak. The company’s tool works by combining graphs generated by a program’s compiler that maps out the flow of the code (such as the control flow graph), according to ShiftLeft’s head of marketing Andrew Fife.
Fife said that the composite graph that ShiftLeft’s SAST tool generates maps all the possible paths through a given application, allowing the tool to check that all appropriate security measures are present along each route.
The ShiftLeft tool looks for “if something is going from point A to point C without going through a point B, which is a check that you do,” said Arun Balakrishnan, ShiftLeft’s director of products. If no point B is found, that means that route is susceptible to security breach.
“If this then translates to the database, it’s a potential SQL injection,” he said. “You could have some string that could have HTML content in it, and it becomes cross-site scripting. If it’s a string that could have a path in it that ends up in the system command, it could be a remote code execution. These are all vulnerabilities that are common in web applications.”
The composite graph also allows users to write their own custom queries to run on the graph, or use templated queries that ShiftLeft provides.
Schwartz said that paid-for tools like ShiftLeft are sometimes also able to have a more sophisticated way of catching more vulnerabilities, by not just looking at isolated chunks of code for vulnerabilities, but by seeing vulnerabilities in the way different parts of code interact.
“If you have two things individually, they may not be bad on their own,” she said. “But if you chain them together, there might be a high probability of something going wrong.”
Using test projects to choose a SAST tool
Schwartz said GitLab sometimes runs different SAST tools against test projects to determine which tools to continue using. This can be a helpful technique for development teams trying to pick an initial SAST tool as well.
“What we do occasionally is we’ll take test projects, run a bunch of the open-source scanners, and be like, ‘This one is the most performant and finding the most issues,’” Schwartz said. “If something new comes out on the market, we might re-benchmark.”
GitLab’s team checks for qualities like scanning speed, the number of false positives and whether the tool is able to catch more vulnerabilities. If a new tool performs better against their benchmarks than one they were previously using, they switch.
For companies that don’t have test projects handy, Schwartz said there are ready-made test projects available online.
“We’ll take test projects, run a bunch of the open-source scanners, and be like, ‘This one is the most performant and finding the most issues.’”
“There are intentionally vulnerable code projects out there,” she said, listing the Juice Shop and WebGoat applications as examples, both dummy websites maintained by the Open Web Application Security Project community and used to demonstrate common security vulnerabilities.
That said, programming language matters for SAST tools, so it’s important to pick a test project that’s written in the same language as your code. Otherwise, results from test runs may not be indicative of the SAST tool’s effectiveness for your application.
Making SAST tools work for your team
Getting a SAST tool that works well with a development team’s workflow also depends on who’s choosing the tool. Schwartz said it’s important to have developer input in the decision-making process.
“In an ideal world, you would have one developer on the engineering team who sort of understands the way the security team thinks,” Schwartz said. “And they could say, ‘Based on the way my team works, I’d like to use this tool.’ Unfortunately, in a lot of cases, you don’t have many engineers who have a passion or inclination to security. So you get the security team saying, ‘Hey, engineering, you have to use this.’ And sometimes that’s OK, but a lot of times the security people don’t understand the development workflow, and then suddenly [the developers] need to do all this implementation and all this other crazy work that interrupts their workflow and causes problems.”
A common complaint among developers who think of SAST as their least favorite four-letter word is that they’re inundated with alerts that, on closer inspection, aren’t real vulnerabilities.
“Unfortunately, all the tools have a high false positive rate,” Schwartz said, adding that a way to avoid wasting development time chasing after alerts all the time is to do a risk analysis, prioritizing parts of the application that are most sensitive to threats.
“It’s like, I store customer information in this database and I have a login page here. These are my two risky areas,” she said. “What you can do is say: ‘I don’t want to scan any parts of my application other than the login page and the pages that allow me to see the customer information or the personally identifiable information. And so suddenly you’re ignoring a whole bunch of areas of your code that, if it gets breached, you don’t really care because you’re going to deprioritize those.”
How DevOps is configured also helps determine whether adopting the SAST tool will integrate smoothly into the development workflow or whether it will feel burdensome.
“Probably the most important thing is your workflow as an engineering team, and where you could insert a scan that would be the least painful for you and the most informative,” Schwartz said. “There’s a balance there. If you’re constantly scanning, that’s a lot of time and resources.”
She stressed the importance of vigilance in pursuing security flaws in code. She mentioned that the Heartbleed Bug discovered in the OpenSSL protocol in 2014, which affected most websites using the secure HTTPS protocol, was the result of a single slip-up by a single developer.
“Basically that person was responsible for introducing this really terrible security bug,” Schwartz said. “No one wants to be that person.”
