DevOps is all about automation. Automated continuous integration and continuous delivery processes allow developers to do more coding and spend less time on deployments. That, in turn, lets teams release code more often — sometimes multiple releases in a single day.
While the adoption of DevOps has benefits for development teams, these automated processes also introduce new risks. Even as DevOps speeds up software development, security hasn’t quite caught up. Until recently, most security checks were manual and occurred after software development was complete.
DevSecOps hopes to change that by automating security processes and making them as fast as the rest of DevOps.
What Is DevSecOps?
“The other angle of DevSecOps, besides allowing security to move at the speed of DevOps, is also to make sure that the speed of DevOps isn’t increasing the attack surface or having a negative effect on the risk landscape,” said Daniel Krivelevich, chief technology officer at Cider Security, which makes security tools for CI/CD processes.
Good DevSecOps is about more than just adding individual security steps into the DevOps process, according to Ada Lin, an IT engineer at zero-trust cloud infrastructure company Teleport. Instead, it requires an attitude and mindset change from the whole company, but especially developers.
“It’s putting an emphasis on shifting left, encouraging engineers to move security to the beginning versus treating it as an afterthought,” Lin said.
The History of DevSecOps
In recent years, application security has become an urgent concern for both the general public and the tech community. Krivelevich said companies used to be much less concerned about security and considered it “much more of a nuisance” — especially compared with the investments they poured into speeding up the software development process.
Security was seen as something companies had to do to avoid negative consequences, like the bad publicity associated with being hacked. But as more customers actively seek out secure products, companies have realized the advantage of not just building a “secure-enough” product, but instead the most secure product they can.
“If my product is more secure than the product of my competition, then this is helpful for my business,” Krivelevich said. “Because people care about the security of the products that they buy.”
This shift has led to the creation of DevSecOps, which tightly binds security concerns into the DevOps workflow. Where DevOps is about creating CI/CD processes and freeing up software developers from doing manual configurations, DevSecOps is about bringing those same advantages to security.
As companies adopt these changes, developers’ roles have changed as well. Here are some skills developers will need to help turn their teams’ DevOps processes into successful DevSecOps processes.
Brainstorming Security Concerns During Project Ideation
Thinking about security starts early. Usually, that means at the design and ideation phase of a project before any code gets written.
At Teleport, teams go through a process known as “request for discussion” to evaluate the potential problems team members can foresee about an upcoming project, Lin said. Engineers at all levels look over a project document, which addresses the project’s purpose and scope, and opens up the floor for discussion.
“That increases the chances of someone recognizing there’s a security concern that should be addressed,” Lin said.
These discussions allow developers to contribute during the early stages of projects, which used to involve only project managers and technical leads. In order to give guidance about the feasibility and work involved in securing potential new features, developers will need to rely on their deep understanding of company technologies and products. Developer viewpoints are particularly beneficial because they provide low-level assessments of technical issues, including potential security issues that need special attention for projects to be successful.
For example, if a customer wants to have role-based access control added to their product, developers can discuss the pros and cons of the security scheme with their team to evaluate different options for implementing it.
Picking the Right Code Scanner
Code-scanning tools are an important part of DevSecOps. They run automatically and can catch well-known security problems in a codebase. These tools, also known as static application security testing tools, are good at quickly finding bad coding patterns that can cause application vulnerabilities.
“If you use more SAST tools, you will be able to catch a wider range of security issues before code is actually pushed to production,” Lin said.
“Part of the big change that the engineering ecosystem has undergone is that there’s a much larger diversity in relation to the technologies being used for development ... Every one of these languages and frameworks has the potential to contain security flaws and vulnerabilities.”
Developers need to keep in mind that these tools are not universal — different programming languages may need different SAST tools that cater to the subtle variations between languages’ security needs.
“Part of the big change that the engineering ecosystem has undergone is that there’s a much larger diversity in relation to the technologies being used for development,” Krivelevich said. “That creates a very dynamic ecosystem of languages and frameworks that are used for development. Every one of these languages and frameworks has the potential to contain security flaws and vulnerabilities.”
Developers need to be involved in the process of finding the right SAST tools for their company’s technology stack, as well as monitoring whether those tools are effective for their languages and applications. For instance, developers can check whether a tool flags too many false positives, like warnings about SQL injection vulnerabilities where they don’t exist. Excessive numbers of false positives will waste a team’s time because each warning needs to be checked out. Developers can compare tools to see which one is most accurate with their programming language and applications.
Quickly Fixing Security Concerns Uncovered by DevSecOps
Developers are responsible for following up and fixing security vulnerabilities uncovered by the DevSecOps process. Even with the old DevOps workflow, developers were already in charge of writing tests and fixing code when tests fail — either during development or during the automated testing phase of DevOps.
“Security also falls under this category of tests that developers run on the code being developed,” Krivelevich said. “Developers need to fix that before they ship the code to production.”
Following up on issues is also part of the feedback loop that gives developers better insights into whether their automated code-scanning tools are working well and whether they should switch to different tools. Are the scanners able to flag lots of missed bugs? Or are they mostly flagging issues that aren’t real problems? If so, those tools risk eating into development time.
“It’s so important to weave those scanners into [the DevSecOps process] in a manner that’s effective and efficient because we don’t want security to slow down engineering,” Krivelevich said. “That’s why we have to have good scanners that are tailored to the specific languages and frameworks in use, that don’t create a lot of false positive results.”
Keeping an Eye on Dependencies
Developers should pay attention to any newly added dependencies to projects, according to Lin. That’s because third-party dependencies can easily introduce security vulnerabilities into codebases. The vulnerability in Java’s logging library Log4j, for example, affected countless companies and organizations when it was discovered last November and forced developers to quickly patch and release code to fix the security hole.
Developers should be extra cautious when using dependencies because they give hackers more avenues for attacking the code.
“It’s also increasing your attack surface, essentially,” Lin said. “There are more security concerns to consider if there’s a new library added.”
But dependencies on third-party and open-source libraries are a fact of life for software development these days, so the best response is keeping an eye out for news about vulnerabilities and fixing them as quickly as possible.
“Questions like, ‘Are all the dependencies that my mission-critical software depends on sustainable components?’ also need attention,” said Christoph Gӧrn, engineering manager at Red Hat. “As software supply chains become deeper, developers need deeper insight into the software stack.”
There are tools developers can use to help track dependencies. Many engineering teams use Dependabot, a tool that maps out an application’s dependency tree and checks whether any dependencies are out of date. Developers can use tools like that to help track dependencies and update the codebase with the most recent version when security vulnerabilities are detected.
A Security Mindset Requires Continuous Learning
DevSecOps works best when all members of the engineering team are engaged with new developments in security. Developers should stay on top of tech news so teams can learn about relevant vulnerabilities and act on them immediately. Any significant delays can result in companies leaving their doors open to attackers.
“For us internally, when [a new vulnerability] comes out, we always start a discussion to make sure the problem is actually addressed,” Lin said.
Engineers should also develop their general understanding of security topics by regularly reading security blogs or getting security certifications. Tech leads should set expectations that team members make security a priority and lead by example, which can help teams build cultures where people are proactive about identifying possible security concerns. It’s the constant practice of taking security seriously, more than anything else, that prevents security checks from getting pushed to the end of development cycles.
“It’s a challenge for everyone to recognize that security is everyone’s responsibility and not just the responsibility of a dedicated team.”
“It’s a challenge for everyone to recognize that security is everyone’s responsibility and not just the responsibility of a dedicated team,” Lin said.
Still, some security practices can’t be automated. Regular internal and external audits should still be a part of every team’s security procedures even when they include manual steps.
Developers can take advantage of company resources to learn more about security as well. Lin said her company provides training for its developers that teach skills like penetration testing and how to conduct post mortems after vulnerabilities are caught in the production environment. While developers may be able to supply the technical skills for building secure applications, it takes the whole team, including management, to create an environment that’s hospitable to good security practices.
“To cope with new challenges, it also requires managers to provide context and create a common understanding of the importance of putting the ‘Sec’ into DevOps,” Gӧrn said. “Ultimately, it is about changing the practices of a company: de-siloing Dev and Ops and de-emphasizing gatekeeper attitudes is a challenge on the human dimension, rather than the technical.”