Deploying code to the production environment is one of the last steps of the software development life cycle, but how engineering teams choose to do it is a decision that should be considered well before any code is ready for release. The blue-green deployment method is one of the options that lets teams fine-tune the transfer of user traffic from old to new versions of an application.
What distinguishes blue-green deployment from other methods of deploying code is the process of creating another copy of the application in production alongside the working version, according to Dave Nielsen, senior director of community and developer advocacy at Harness, a software delivery orchestration platform.
What Is Blue-Green Deployment?
“The blue one would be your newer version and your green would be the one that was the current version,” Nielsen said. The development team can then redirect a portion of the application’s user traffic to the blue version instead of the green version. Once the system is confirmed to be stable, the remainder of the user traffic can be shifted to the blue version, which in effect becomes the new green version.
As more companies adopt cloud computing and microservice-style architecture, it’s gotten easier for development teams to use the blue-green method to release code. Both of those technologies are well suited for the blue-green style of infrastructure-heavy deployments, where releases go smoother when developers ship small sections of code at a time.
Here, a few experts shared their experiences using blue-green deployment and the unique advantages and challenges it offers.
Comparing Canary Deployments to Blue-Green Deployments
Using canary deployments can sometimes make more sense than using blue-green deployments, Nielsen said. Blue-green deployment creates a new production environment alongside the existing production environment, but canary deployments just push the new code to the existing production environment. It’s less resource heavy because it only affects the portion of the codebase that’s different from the existing code.
But the two different methods share some common advantages. Both allow developers to direct a configurable portion of user traffic toward a new version of the application while the existing application is still running. Developers can test and work out kinks in the new application version without a majority of users feeling the impact from bugs. Then, once those problems have been fixed, the remaining user traffic can be transitioned over to the new version, providing users with a seamless deployment experience.
Blue-green methodology, however, also has advantages and challenges unique to its style of deployment.
Blue-Green Offers Developers a Hands-Off Approach to Testing Deployments
Changes in the way development teams typically architect and host software has made blue-green deployment a much more viable option than in the past, Marshall Greer, CTO at digital media SaaS company Greenfly, said.
It used to be that spinning up additional servers was a huge undertaking, consisting of complex configurations and many steps of trial and error. Because managing server infrastructure was so difficult, companies rarely allocated entirely separate servers for the sole purpose of testing deployment. Instead, development teams would use feature flags to change the logic within the application code to allow a certain percentage of user traffic to flow through the new version while the majority of user traffic stayed within the existing production version.
“If you wanted to have different versions of your code made available to customers, you needed to include that business logic in your code itself,” Greer said.
“We increasingly want to be able to treat infrastructure like livestock and less like pets ... And blue-green kind of makes that more possible.”
While it worked well, the disadvantage of that method was it relied on developers to create the configuration changes in the application. Essentially, it meant developers would be spending time on writing deployment configurations into the code when they could have been using the time to build out more features for customers. It added more responsibilities onto the plates of developers who were already stretched thin.
But now as more development teams have switched from hosting applications on their own physical servers to using cloud computing services, they have gained the ability to quickly and easily spin up additional server resources with the click of a button, making it much easier for developers to add and remove server resources. That gives developers the option to use servers to test out user traffic instead of embedding that logic all within the application code. In turn, these server-based deployments free up developers’ time from having to manage a lot of deployment configurations in the code.
“We increasingly want to be able to treat infrastructure like livestock and less like pets,” Greer said, referring to the hands-off approach offered by easily spinning up additional servers. “And blue-green kind of makes that more possible.”
Blue-Green Deployments Are Optimized for Microservices
Although blue-green deployments can help free up developers from managing the code release process, there are certain types of applications that don’t work well with the blue-green methodology. Monolithic applications are usually not worth switching over to a blue-green model, for example.
“If you have, effectively, a single instance of your codebase managing various applications all within the same codebase, then you’re not going to be able to independently deploy new versions of services,” Greer said.
That’s a problem for blue-green deployment. It means the entirety of the application code must be copied to a new server for each release, regardless of where the code changes occurred or how limited the changes were. It makes the entire process overly cumbersome and also prevents the development team from releasing different changes at the same time.
Greer gave the example of an application that has five different services. If the application were built following a microservice-style architecture, each service would exist independently of the other and run on a separate server. If all five services had code changes that needed to be released at the same time, developers could simply spin up five additional servers alongside the existing ones and direct user traffic to them.
“If I have other services that may have a dependency on that fifth service, they’re still able to communicate with it, they’re not even aware of the underlying application changes,” he said.
If the application were built as a monolith, on the other hand, those changes would not be able to be released and tested independently. That could add significant complexity and cut into the time it takes to deploy services using the blue-green method.
Blue-Green Deployment Is All About Routing
The concept behind blue-green deployment is quite simple: Fundamentally, it’s just hosting two versions of an application in production and funneling different levels of user traffic to each version. How the logic of directing that user traffic is accomplished plays a large role in how blue-green deployments are carried out, and it can be quite complex technically.
“Whatever solution you’re using to manage the routing of requests is a key and a necessary component of blue-green deployments,” Greer said.
For a lot of development teams, that means using some kind of orchestration tool for code containers to manage the process of routing requests. Development teams might choose to host their applications in different virtual machines or containers, Nielsen said, then use a tool like Kubernetes to deploy new clusters whenever code from a service needs to be released. Load balancers help to direct user traffic from the server containing the old version of the code to the server with the new version.
“So, you would tell your load balancer that if anything comes into your production URL, it goes to the green cluster of IP addresses, and anything going to the nonproduction URL goes to the second cluster,” Nielsen said. “And then all you do when you’re happy with your new environment is you just switch the load balancer to point to the opposite IP addresses.”
Beware of Breaking Changes When Using Blue-Green Deployment
Another scenario that blue-green deployment doesn’t handle well is when changes are made in a database schema that causes corresponding code to break. That can occur whenever developers are making adjustments to the structure of a database, for example, by deleting an existing table or adding additional columns to a table.
Problems can happen when a blue version of the code includes a schema change that the green version hasn’t accounted for. For instance, if the blue version is pushed out to production with a command to delete an existing table, the schema change can immediately break the logic for the green version. That’s because both versions share the same database, so the code in the green version that still references the deleted database table won’t be able to work properly.
“I have heard instances where companies will deploy a new version of the app that includes schema changes, and that breaks the existing version of the app,” Greer said. A common quick fix to the problem, according to Greer, is having development teams roll up all the traffic to the new version right away to bring the database back into sync with the code.
Although these database breaking changes can be a problem for blue-green deployments, they have become less common as more companies shift to using schemaless databases instead of relational databases, Greer said.
“But still, even in the NoSQL world, sometimes you need to store your data in ways that are not backwards compatible,” he said. “And when that happens, you have to deploy a new version of your application logic all at once in order to match it up with the schema changes.”
Complex Deployments Need Strong DevOps Teams
Development teams should plan their deployment strategies carefully. Even though any application that’s hosted on the cloud and built using a microservice architecture may appear to be an ideal candidate for blue-green deployment, that may not necessarily be the case, Greer said.
Spinning up additional servers on the cloud is simple, thanks to services like Kubernetes, but the logic involved in routing user traffic to different servers and juggling different versions of services across an application can get quite complex. At the very least, development teams need to invest in robust DevOps teams before they can expect to handle the logistics of blue-green deployment well.
“In some ways, you’re exchanging application complexity with infrastructure complexity because then you have to manage different versions within your infrastructure,” Greer said.
But perhaps more importantly, development teams should ask themselves whether it’s worthwhile to use a complex deployment strategy like blue-green deployment. Would the application get any benefits from being able to test multiple services in the production environment side by side, or would it instead be injecting even more complexity into the system?
“In some ways, you’re exchanging application complexity with infrastructure complexity.”
“One of the mistakes our industry has historically made is you hear about some new technology or some new approach, and you have a lot of hype that builds around it,” Greer said. He said it may not be the best idea to introduce blue-green deployment into applications that “don’t have enough business complexity with version management to necessitate the complexity that you need to have to run your deployment model.”
But if your application is already hosted on the cloud and built using microservice-style architecture, and your engineering team wants to have control over how user traffic is transitioned between versions of the application during release, blue-green deployment could be a great method that frees up developers’ time from having to maintain feature flags in the code. Investing in DevOps engineers and processes can also help teams to reap its full benefits.