As a developer, you’ve likely used environment variables in the command line or shell scripts, but have you used them as a way of configuring your Python applications?
This guide will show you all the code necessary for getting, setting, and loading environment variables in Python, including how to use them for supplying application config and secrets.
Not familiar with environment variables? Check out our ultimate guide for using environment variables in Linux and Mac.
Before digging into how to use environment variables in Python, it's important to understand why they're arguably the best way to configure applications. The main benefits are:
Environment variables have the additional benefit of abstracting from your application how config and secrets are supplied.
Finally, environment variables enable your application to run anywhere, whether it's for local development on macOS, a container in a Kubernetes Pod, or platforms such as Heroku or Vercel.
Here are some examples of using environment variables to configure a Python script or application:
When a Python process is created, the available environment variables populate the os.environ object which acts like a Python dictionary. This means that:
Now that you know how environment variables in Python are populated, let's look at how to access them.
Environment variables in Python are accessed using the os.environ object.
The os.environ object seems like a dictionary but is different as values may only be strings, plus it's not serializable to JSON.
You've got a few options when it comes to referencing the os.environ object:
I personally prefer version 3 as it's more succinct, but will stick to using os.environ for this article.
Accessing a specific environment variable in Python can be done in one of three ways, depending upon what should happen if an environment variable does not exist.
Let's explore with some examples.
If your app should crash when an environment variable is not set, then access it directly:
For example, an application should fail to start if a required environment variable is not set, and a default value can't be provided, e.g. a database password.
If instead of the default KeyError exception being raised (which doesn't communicate why your app failed to start), you could capture the exception and print out a helpful message:
You can have a default value returned if an environment variable doesn't exist by using the os.environ.get method and supplying the default value as the second parameter:
If the variable doesn't exist and you use os.environ.get without a default value, None is returned
You may need to check if an environment variable exists, but don't necessarily care about its value. For example, your application can be put in a "Debug mode" if the DEBUG environment variable is set.
You can check for just the existence of an environment variable:
Or check to see it matches a specific value:
Setting an environment variable in Python is the same as setting a key on a dictionary:
What makes os.environ different to a standard dictionary, is that only string values are allowed:
In most cases, your application will only need to get environment variables, but there are use cases for setting them as well.
For example, constructing a DB_URL environment variable on application start-up using DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, and DB_NAME environment variables:
Another example is setting a variable to a default value based on the value of another variable:
If you need to delete a Python environment variable, use the os.environ.pop function:
To extend our DB_URL example above, you may want to delete the other DB_ prefixed fields to ensure the only way the app can connect to the database is via DB_URL:
Another example is deleting an environment variable once it is no longer needed:
To view all environment variables:
The output of this command is difficult to read though because it's printed as one huge dictionary.
A better way, is to create a convenience function that converts os.environ to an actual dictionary so we can serialize it to JSON for pretty-printing:
You might be surprised to learn it's best to avoid providing default values as much as possible. Why?
Default values can make debugging a misconfigured application more difficult, as the final config values will likely be a combination of hard-coded default values and environment variables.
Relying purely on environment variables (or as much as possible) means you have a single source of truth for how your application was configured, making troubleshooting easier.
As an application grows in size and complexity, so does the number of environment variables.
Many projects experience growing pains when using environment variables for app config and secrets because there is no clear and consistent strategy for how to manage them, particularly when deploying to multiple environments.
A simple (but not easily scalable) solution is to use a .env file to contain all of the variables for a specific environment.
Then you would use a Python library such as python-dotenv to parse the .env file and populate the os.environ object.
Now save the below to a file named .env (note how it's the same syntax for setting a variable in the shell):
Then save the following to dotenv-test.py:
Then run dotenv-test.py to test the environment variables are being populated:
While .env files are simple and easy to work with at the beginning, they also cause a new set of problems such as:
Doppler provides an access controlled dashboard to manage environment variables for every environment with an easy to use CLI for accessing config and secrets that works for every language, framework, and platform.
Creating a config specific data structure abstracts away how the config values are set, what fields have default values (if any), and provides a single interface for accessing config values instead of os.environ being littered throughout your codebase.
Below is a reasonably full-featured solution that supports:
To try it out, save this code to config.py:
The Config object exposed in config.py is then used by app.py below:
Make sure you have the .env file still saved from earlier, then run:
Awesome work! Now you know how to use environment variables in Python for application config and secrets.
Although we're a bit biased, we encourage you to try using Doppler as the source of truth for your application environment variables, and it's free to get started with our Community plan (unlimited projects and secrets).
We also have a tutorial for building a random Mandalorion GIF generator that puts into practice the techniques shown in this article.
Hope you enjoyed the post and if you have any questions or feedback, we'd love to chat with you over in our Community forum.
Photo by Hitesh Choudhary on Unsplash.