AWS Secrets Manager is an excellent service for managing sensitive information like database credentials, certificates, passwords, and tokens in the cloud. While infrastructure as code (IaC) tools like Terraform can integrate with secret management services like Secrets Manager, KMS, or Parameter Store, Terraform’s native secrets management has some limitations.

This article discusses best practices for secret management in Terraform. We cover how Terraform can be configured to create and retrieve secrets using Secrets Manager and discuss the challenges of Terraform’s native secrets management approach. We also look at how good third-party tools can address Terraform’s limitations and provide an example of how such a tool can streamline secret management for multiple platforms.

Summary of secrets supported in Terraform

Let’s take a quick look at the different secrets that can be used within Terraform and their use cases in an AWS environment.

Summary of secret management best practices in Terraform

The table below gives an overview of the best practices to follow while managing secrets in Terraform. We will explain each in detail in the following section.

Best practices for secret management in Terraform

Now that you’ve gotten a brief idea about the best practices, let’s take a deeper dive into each one, with examples.

Avoid storing secrets in plain text

Storing secrets as plain text inside code exposes the secrets to anyone having access to your version control system when you push the code. This can be dangerous if the repository is public and can be avoided by using secrets management services like AWS Secrets Manager, AWS Parameter Store, or Hashicorp Vault.

Here’s an example of storing secrets using AWS Secrets Manager in Terraform.

1. Create a main.tf file and add the following code to create your secret.

1terraform {
2 required_providers {
3  aws = {
4  source = "hashicorp/aws"
5  version = "~> 4.16"
6 }
7}
8
9
10 required_version = ">= 1.2.0"
11}
12
13
14provider "aws" {
15 region = "ap-south-1"
16}
17
18
19resource "aws_secretsmanager_secret" "db_password" {
20 name = "dev/mysql/password"
21}
22
23
24resource "aws_secretsmanager_secret_version" "db_password_version" {
25 secret_id = aws_secretsmanager_secret.db_password.id
26 secret_string = var.db_password
27}

2. Create a variables.tf file as follows:

1variable "db_password" {
2 description = "Database administrator password"
3 type = string
4 sensitive = true
5}

3. Now, use terraform plan and pass the secret as a variable to review your changes:

1➜ terraform plan -var db_password=securevalue -out tfplan
2
3Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
4the following symbols:
5  + create
6
7Terraform will perform the following actions:
8
9  # aws_secretsmanager_secret.db_password will be created
10  + resource "aws_secretsmanager_secret" "db_password" {
11      + arn                            = (known after apply)
12      + force_overwrite_replica_secret = false
13      + id                             = (known after apply)
14      + name                           = "dev/mysql/password"
15      + name_prefix                    = (known after apply)
16      + policy                         = (known after apply)
17      + recovery_window_in_days        = 30
18      + rotation_enabled               = (known after apply)
19      + rotation_lambda_arn            = (known after apply)
20      + tags_all                       = (known after apply)
21    }
22
23  # aws_secretsmanager_secret_version.db_password_version will be created
24  + resource "aws_secretsmanager_secret_version" "db_password_version" {
25      + arn            = (known after apply)
26      + id             = (known after apply)
27      + secret_id      = (known after apply)
28      + secret_string  = (sensitive value)
29      + version_id     = (known after apply)
30      + version_stages = (known after apply)
31    }
32
33Plan: 2 to add, 0 to change, 0 to destroy.

4. Finally, perform terraform apply to deploy the configuration.

1➜ terraform apply tfplan                                 
2aws_secretsmanager_secret.db_password: Creating...
3aws_secretsmanager_secret.db_password: Creation complete after 1s [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY]
4aws_secretsmanager_secret_version.db_password_version: Creating...
5aws_secretsmanager_secret_version.db_password_version: Creation complete after 0s [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|8C202066-3580-4B46-85D4-5008105BFD62]
6
7Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

Use the Terraform data module to retrieve secrets

The Terraform data module allows Terraform to use external information in its configuration. This can be done by specifying the appropriate provider, such as an AWS Secrets Manager data source.

The following example demonstrates how you can use the AWS Secrets Manager data source to retrieve secrets into your Terraform configuration from AWS.

1. Create a main.tf file and add the following code specifying the Secrets Manager data module:

1terraform {
2 required_providers {
3  aws = {
4  source = "hashicorp/aws"
5  version = "~> 4.16"
6 }
7}
8
9
10 required_version = ">= 1.2.0"
11}
12
13
14provider "aws" {
15 region = "ap-south-1"
16}
17
18
19data "aws_secretsmanager_secret" "db_password" {
20 name = "dev/mysql/password"
21}
22
23
24data "aws_secretsmanager_secret_version" "db_password_version" {
25 secret_id = data.aws_secretsmanager_secret.db_password.id
26}
27
28
29// Example RDS instance using the secrets
30resource "aws_db_instance" "default" {
31 allocated_storage = 10
32 db_name = "mydb"
33 engine = "mysql"
34 engine_version = "5.7"
35 instance_class = "db.t3.micro"
36 username = var.db_username
37 password = data.aws_secretsmanager_secret_version.db_password_version.secret_string
38 parameter_group_name = "default.mysql5.7"
39 skip_final_snapshot = true
40}

2. If you have stored secrets as JSON, you can use the following method instead, which utilizes jsondecode:

1password = jsondecode(data.aws_secretsmanager_secret_version.db_password_version.secret_string)["password"]

Pass secrets to Terraform as input variables, variable definition files, or environment variables

Secrets can be passed to Terraform as input variables, variable definition files, or environment variables.

Input variables

Users can pass the secrets with the -var command line option while running Terraform commands. The following example shows how users can pass input variables during the Terraform plan state and apply the Terraform configuration to deploy resources.

1➜ terraform plan -var db_password=secretvalue -out tfplan
2aws_secretsmanager_secret.db_password: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY]
3aws_secretsmanager_secret_version.db_password_version: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|FBBB224C-1C04-4544-A59C-5BFF7089B019]
4
5Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
6the following symbols:
7-/+ destroy and then create replacement
8
9Terraform will perform the following actions:
10
11  # aws_secretsmanager_secret_version.db_password_version must be replaced
12-/+ resource "aws_secretsmanager_secret_version" "db_password_version" {
13      ~ arn            = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY" -> (known after apply)
14      ~ id             = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|FBBB224C-1C04-4544-A59C-5BFF7089B019" -> (known after apply)
15      ~ secret_string  = (sensitive value) # forces replacement
16      ~ version_id     = "FBBB224C-1C04-4544-A59C-5BFF7089B019" -> (known after apply)
17      ~ version_stages = [
18          - "AWSCURRENT",
19        ] -> (known after apply)
20        # (1 unchanged attribute hidden)
21    }
22
23Plan: 1 to add, 0 to change, 1 to destroy.
24
25──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
26
27Saved the plan to: tfplan
28
29To perform exactly these actions, run the following command to apply:
30    terraform apply "tfplan"
31➜ terraform apply tfplan
32aws_secretsmanager_secret_version.db_password_version: Destroying... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|FBBB224C-1C04-4544-A59C-5BFF7089B019]
33aws_secretsmanager_secret_version.db_password_version: Destruction complete after 0s
34aws_secretsmanager_secret_version.db_password_version: Creating...
35aws_secretsmanager_secret_version.db_password_version: Creation complete after 1s [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|AF6DCE17-9D9E-41E0-A465-8CA2D5A63BB2]
36
37Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

Variable definition files

Users can enter the secrets inside a .tfvars file and specify this on the command line using the -var-file argument. However, this requires users to be careful to not push the tfvars file to the version control system and maintain it themselves.

Let’s go through an example by creating a secrets.tfvars file with the following content:

1db_password="securevalue"

Apply the configuration to deploy the changes using the following command:

1terraform plan -var-file=secrets.tfvars -out tfplan
2terraform apply tfplan

Environment variables

Users can export secrets in the command line as TF_VAR_ followed by the variable name before running the Terraform commands. The following example shows how users can use environment variables to pass secrets to Terraform:

1➜ export TF_VAR_db_password=securevalue
2➜ terraform plan -out tfplan
3aws_secretsmanager_secret.db_password: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY]
4aws_secretsmanager_secret_version.db_password_version: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|9F82C9E6-438A-4576-8D5E-117FB563C9E2]
5
6Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with
7the following symbols:
8-/+ destroy and then create replacement
9
10Terraform will perform the following actions:
11
12  # aws_secretsmanager_secret_version.db_password_version must be replaced
13-/+ resource "aws_secretsmanager_secret_version" "db_password_version" {
14      ~ arn            = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY" -> (known after apply)
15      ~ id             = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|9F82C9E6-438A-4576-8D5E-117FB563C9E2" -> (known after apply)
16      ~ secret_string  = (sensitive value) # forces replacement
17      ~ version_id     = "9F82C9E6-438A-4576-8D5E-117FB563C9E2" -> (known after apply)
18      ~ version_stages = [
19          - "AWSCURRENT",
20        ] -> (known after apply)
21        # (1 unchanged attribute hidden)
22    }
23
24Plan: 1 to add, 0 to change, 1 to destroy.
25
26──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
27
28Saved the plan to: tfplan
29
30To perform exactly these actions, run the following command to apply:
31    terraform apply "tfplan"
32➜  secrets manager tf terraform apply tfplan
33aws_secretsmanager_secret_version.db_password_version: Destroying... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|9F82C9E6-438A-4576-8D5E-117FB563C9E2]
34aws_secretsmanager_secret_version.db_password_version: Destruction complete after 0s
35aws_secretsmanager_secret_version.db_password_version: Creating...
36aws_secretsmanager_secret_version.db_password_version: Creation complete after 0s [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|FBBB224C-1C04-4544-A59C-5BFF7089B019]
37
38Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

Store Terraform state in a remote backend instead of a local state file

Since a Terraform state file contains sensitive information about your resource configurations, it is recommended to store the state files in a remote backend like S3 bucket and avoid pushing them to the version control system.

To create a new S3 backend for your Terraform project, follow these steps:

1. Add the following code inside your main.tf file after specifying your bucket name:

1terraform {
2 backend "s3" {
3  bucket = "terraformstate-storage-bucket"
4  key = "prod/terraform.tfstate"
5  region = "ap-south-1"
6 }
7}

2. Next, perform terraform init to initialize your remote backend:

1➜ terraform init
2Initializing the backend...
3Do you want to copy existing state to the new backend?
4Pre-existing state was found while migrating the previous "local" backend to the newly configured "s3" backend. No existing state was found in the newly configured "s3" backend. Do you want to copy this state to the new "s3" backend? Enter "yes" to copy and "no" to start with an empty state.
5Enter a value: yes
6Successfully configured the backend "s3"! Terraform will automatically use this backend unless the backend configuration changes.
7Initializing provider plugins...
8- Reusing previous version of hashicorp/aws from the dependency lock file
9- Using previously-installed hashicorp/aws v4.67.0
10Terraform has been successfully initialized!
11You may now begin working with Terraform. Try running "terraform plan" to see any changes that are required for your infrastructure. All Terraform commands should now work.
12If you ever set or change modules or backend configuration for Terraform, rerun this command to reinitialize your working directory. If you forget, other commands will detect it and remind you to do so if necessary.

Note that it is recommended to have versioning enabled on your S3 bucket so that you can recover old state files in case.

Use tools that can inject secrets dynamically for team collaboration

Integration with tools like Doppler can help inject secrets from multiple providers dynamically into your Terraform code, helping keep a single source of truth for the secrets across your projects.

The following example demonstrates how users can make use of Doppler CLI to dynamically inject secrets into Terraform.

1. First, make sure you have Doppler CLI installed on your system and configure it by doing doppler login and doppler setup to grant Doppler API access to the required secrets.

2. Next, integrate Doppler with AWS Secrets Manager. Doppler supports integration with multiple services; check out this article to learn how to integrate Doppler with AWS Secrets Manager.

3. Once you’ve finished the integration, create your secret from the Doppler dashboard. You could choose to generate a random secret value as follows for added security:

4. Finally, run the following command to inject the secret inside your Terraform configuration:

1doppler run --name-transformer tf-var -- terraform plan -out tfplan

Let’s verify this by running the command from the terminal.

1➜ doppler run --name-transformer tf-var -- terraform plan -out tfplan
2aws_secretsmanager_secret.db_password: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY]
3aws_secretsmanager_secret_version.db_password_version: Refreshing state... [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|79D3EC33-FE5C-4F8E-B20E-F822214C4199]
4
5Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
6-/+ destroy and then create replacement
7
8Terraform will perform the following actions:
9
10  # aws_secretsmanager_secret_version.db_password_version must be replaced
11-/+ resource "aws_secretsmanager_secret_version" "db_password_version" {
12      ~ arn            = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY" -> (known after apply)
13      ~ id             = "arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|79D3EC33-FE5C-4F8E-B20E-F822214C4199" -> (known after apply)
14      ~ secret_string  = (sensitive value) # forces replacement
15      ~ version_id     = "79D3EC33-FE5C-4F8E-B20E-F822214C4199" -> (known after apply)
16      ~ version_stages = [
17          - "AWSCURRENT",
18        ] -> (known after apply)
19        # (1 unchanged attribute hidden)
20    }
21
22Plan: 1 to add, 0 to change, 1 to destroy.
23
24─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
25
26Saved the plan to: tfplan
27
28To perform exactly these actions, run the following command to apply:
29    terraform apply "tfplan"
30➜  secrets manager tf terraform apply tfplan
31aws_secretsmanager_secret_version.db_password_version: Destroying... [id=arn:aws:secretsmanager:ap-south-1:{accountNumber}:secret:dev/mysql/password-zGdSnY|79D3EC33-FE5C-4F8E-B20E-F822214C4199]
32aws_secretsmanager_secret_version.db_password_version: Destruction complete after 0s
33aws_secretsmanager_secret_version.db_password_version: Creating...
34aws_secretsmanager_secret_version.db_password_version: Creation complete after 0s [id=arn:aws:secretsmanager:ap-south-1:123123123123:secret:dev/mysql/password-zGdSnY|9F82C9E6-438A-4576-8D5E-117FB563C9E2]
35
36Apply complete! Resources: 1 added, 0 changed, 1 destroyed.

As you can see, Doppler dynamically injected the secrets into the Terraform configuration while keeping the secret as a sensitive value.

Challenges of Terraform’s native secrets management approach

Terraform’s native secrets management has the following drawbacks:

Insecure by default: By default, secrets are stored as plain text in Terraform. Users have to encrypt manually to create a secure approach.
Limited scope: Terraform’s secrets capability is limited to Terraform configurations only; it cannot be used to store or manage secrets outside Terraform.
Tightly coupled: The secrets created using Terraform are tightly coupled to Terraform configurations. Terraform’s native approach does not provide a centralized way to store secrets across multiple accounts or environments.
Version control issues: Since Terraform stores secrets as plain text by default, they need to be excluded from version control. This causes issues with collaboration and auditing.

How Doppler helps overcome Terraform’s limitations

Doppler can help improve Terraform by providing the following advantages:

Secure by default: All the secrets stored using Doppler are encrypted at rest and in transit. Users do not have to manually encrypt secrets.
Broad scope: Doppler is a dedicated secrets management solution that can help store and manage secrets across all infrastructure, applications, and teams.
Decoupled from configurations: Secrets in Doppler are decoupled from Terraform configurations and can be used across multiple accounts or environments. This makes configurations more robust and reusable.
Better version control: Since Doppler stores secrets externally, Terraform configurations can be stored within version control as usual, helping improve collaboration and auditing.
Support for multiple providers: Doppler integrates with various secret services and cloud providers beyond the limited set supported by Terraform natively.
Advanced features: Doppler provides additional features like secret rotation, auditing, access control, etc., helping achieve a robust secret management solution.
Simple integration: Doppler integrates easily with Terraform using its Terraform provider. Secrets can be fetched directly from Doppler into Terraform configurations.

Conclusion

This article provides an in-depth look at using AWS Secrets Manager with Terraform. While Terraform can be used to provision and manage secrets in Secrets Manager, Terraform’s native approach has some limitations. We described these limitations in detail and went through best practices and security recommendations that can help improve Terraform’s secret management capability.

The article also highlights how combining Terraform with tools like Doppler can make secret management more efficient and secure. Since Doppler integrates with multiple secret management services, it can be a reliable option for managing secrets across different cloud providers, empowering teams to scale their deployments efficiently while maintaining security and control over sensitive data.

If you’re interested in learning more about Doppler, check out the free team trial for 14 days without a credit card or request a demo to try Doppler for the enterprise.

Enter the new era of secrets management

Trusted by the world’s best DevOps and security teams. Doppler is the secrets manager developers love.

Start For Free Get Demo