Sooner or later, we all have to deal with environment variables in our Compose files. They can become a pain, especially if we don’t know how to use them properly. Here’s everything I know about environment variables and how to make using these variables easy and, above all, secure.
How to Use Environment Variables in Docker
Docker Compose allows us to pass environment variables in via command line or to define them in our shell. However, it’s best to keep these values inside the actual Compose file and out of the command line. “Why?” you may ask.
Because this way, we don’t have to remember all of the environment variables we use every time we deploy our container. By storing them in the Compose file, we maintain consistency during our builds.
There are several ways to do this.
Use the Environment Option
Using the Compose environment option allows us to declare environment variables and their values inside our Compose file, like this.
This is the easiest, quickest way to store environment variables inside Compose files. However, it has a huge drawback and it’s related to security. Can you guess what it is?
That’s right.
Storing the values of your environment variables in the Compose file will — 95 percent of the time — go straight to source control and that’s a huge security risk. Luckily, we have an alternative: using an external file to store our environment variables.
Use a .env File
The main advantage of using an external file for your environment variables is that you can keep said file out of your source control. After all, no one enjoys having their passwords, API keys or other super-secret information all over the internet.
.env files are plain text files, which we use for configuration. Remember, because the file name begins with a ‘.’, they remain hidden to the system.
Tip: To list hidden files, you can use the ls -a command on Linux, or the dir /a:h command on Windows.
You must create .env files at the root of your project, which is also where your docker-compose.yml file should be.
We can declare and assign variables in our .env file. Name the variables however you want since we’ll only access their values.
Here’s my .env file:
You can also create and populate your .env file from the command line, by using the Linux cat command:
Tip: Remember not to leave any spaces between the = sign and the value assigned to your variable, as they will be added to the string.
Now that we have our variables stored in our .env file, let’s use them in our Compose file. Now’s the time to use string interpolation (that’s a fancy name for using the notation ${string}) to assign the values of our .env variables to the environment variables in the Compose file.
As you can see, we maintain the environment option and simply assign our external values to the Compose environment variables.
To check that everything’s working properly, run the following command: docker-compose up.
Tip: You can check which values are assigned to the environment variables by running the following command (in a different terminal): docker-compose config.
Environment Variables in Docker Compose
Environment Variable Priority
Something very important we must keep in mind is the priority used by Compose to choose its environment value. What does this mean?
If we declare the same environment variable in several files (e.g, in the Compose file and in the external .env file) with different values, Compose will use the value of the variable declared in the Compose file. Why? Because, depending on where the variable is declared, Compose gives that variable a higher or lower priority. Here’s the order, ranking from highest priority to lowest:
-
Compose file
-
Shell environment variables
-
Environment file
-
Dockerfile
-
Variable is not defined
If, for some reason, Compose is picking up and assigning a value you weren’t expecting, this is probably the cause. Make sure to have all your variables declared exactly where you want.
Use the env_file Option
In the previous section, we talked about plain .env files, but we never got around to using named .env files. If we do want our .env file to have a name, like secret-stuff.enf, Compose has a nifty little option named env_file.
This option allows us to tell Compose which .env file it has to look for, as opposed to its default behavior, which is to look for an unnamed .env file. This is how we use the env_file option.
As you can see, we added the env_file option, which points to a file named secret-stuff.env file. All that’s left is to rename our previous .env file to secret-stuff.env.
You may have noticed the environment option is no longer present in our Compose file. This is because using the env_file option raises a problem — one that gave me quite a headache!
To assign values declared in an external named .env file to Compose variables requires said file to be defined in the primary Compose service. This is related to the order that Compose follows when performing operations. However, we have no primary service, since we are only using a db service. Therefore, if we try to deploy our Compose file, it will complain our variables are undefined, and substitute them with blank strings.
You don’t have to take my word for it, go ahead and try it! Run docker-compose up and see what happens.
So, how do we solve this problem? Here’s what I figured out:
If we remove the environment option from the Compose file, upon deployment, Compose will search for the specified secret-stuff.env file, something it doesn’t do when the environment option is present. Problem solved!
Bear in mind though, since we don’t have the environment option any longer, we must declare the environment variables directly in the secret-stuff.env file, like this:
Once again, to check that everything works properly, run: docker-compose up.
That’s all, folks! By now you’ve (hopefully!) learned the different ways of securely dealing with environment variables in Compose files.
