Automate and govern GitHub Runners with Nix, Terraform, and Doppler’s Change Requests, Policies, and Analytics

In this post, I will walk you through how I have set up this environment and provide you with the tools and code to deploy into your own environment. I will also show how the new Doppler Analytics Dashboard, Change Requests, and Change Request Policies features allow for quick insights and structured changes.

The blueprint: Architecting secure and automated macOS runner farm

Before we roll up our sleeves and delve into the configuration specifics, let's outline the architecture of our automated and secure GitHub Actions runner deployment. Our goal is a system where macOS runners can be provisioned consistently, managed centrally, and where all secrets are handled with robust governance and visibility. This blueprint leverages the unique strengths of Nix with Home Manager for environment consistency, Terraform for orchestration, and Doppler for state-of-the-art secret operations.

Component inventory and their roles

1 -macOS Nodes (Mac Minis): Our target machines that host the GitHub actions runners. They run standard macOS, but this can be modified to any Linux distribution of our choice.

2 - Nix + Home Manager: Deployed on each Mac Mini, these tools ensure that every runner operates in an identical, declaratively defined user environment. This means consistent versions of Git, Curl, Terraform, and any other necessary build tools, eliminating the "works on my machine" problem for CI-based jobs.

3 - Doppler: The centralized secrets management platform will securely store:

• SSH Private Keys: Used by Terraform on the deployment machine to connect to and configure each Mac Mini.
• GitHub Actions Runner Registration Tokens: Unique, short-lived tokens required to register each new runner instance with GitHub
• Crucially, we will also look into Doppler's Change Request Policies, how it can provide an approval workflow for updating these critical secrets, and how their Analytics Dashboard offers vital insights into their usage.

4 - Deployment Machine: This is a central machine (it could be your local workstation or a dedicated CI/CD orchestrator) where Terraform commands are executed. It has the Doppler CLI installed and configured.

5 - Terraform: The Infrastructure as Code (IaC) tool used on the deployment machine to:

• Fetch SSH keys and GitHub runner tokens from Doppler.
• Connect to each Mac Mini.
• Execute a script on the Mac Mini to download, configure, and install the GitHub Actions runner software as a persistent service.

Ensuring consistent environments across nodes

Before we automate the GitHub runner installation, our target Mac Mini (Servers) must provide a consistent and correct environment for our CI Jobs. This is where Nix and Home Manager shine on macOS. Our Home Manager configuration defines the exact packages available to the user environment.

After applying this configuration on each Mac Mini with ⁠home-manager switch --flake 'path:/path/to/your/Home-Manager#user@system', user ⁠0x53c will have these tools consistently in their PATH, sourced from the Nix store. This forms the reliable foundation for our GitHub Actions runner.

Securely managing credentials with Doppler

Our automation relies on two main types of secrets:

1. SSH Private Keys: For Terraform to access the Mac Mini.
2. GitHub Actions Runner Registration Tokens: For each runner instance.

These are stored in Doppler. On our deployment machine, we ensure the ⁠DOPPLER_TOKEN environment variable is set, granting Terraform access via the Doppler provider.

Change Request Policies: The first line of defense

Doppler's Change Request Policies features transform how teams are able to manage secret modifications. The feature moves organizations from reactive to proactive security by implementing structured workflows for changes to secrets. Every single modification to sensitive credentials will follow pre-defined proposal > review > approval workflow.

Real-world application

I would strongly recommend taking into account the following scenarios where implementing a change request workflow could play a crucial role in preventing potential security incidents and enhancing overall operational integrity:

1. The process of rotating API keys should be strictly regulated and managed to ensure that no modifications occur without explicit coordination and agreement among all relevant teams and stakeholders. By enforcing this protocol, we can significantly mitigate the risk of unnecessary breaking changes that might disrupt services and compromise security. This structured and methodical approach ensures that all involved parties are informed and engaged in the decision-making process and enhances our overall security posture and operational stability. It creates a transparent environment where any changes are clearly communicated and understood by everyone, thereby minimizing potential vulnerabilities.
2. Emergency Breakglass procedures must be thoroughly documented and accounted for, alongside the implementation of a context-aware audit trail. This ensures that in the event of a security incident or emergency, there is a clear and actionable process in place that can be executed swiftly and effectively. The inclusion of a context-aware audit trail adds an additional layer of security by allowing us to track and review actions taken during such emergencies, ensuring accountability and aiding in the analysis of any incidents that may arise. This dual focus on preparedness and accountability is essential for maintaining the integrity and security of our systems.

Demonstration integration

In our GitHub Runner automation, we're managing multiple sensitive credentials that require strict change control. From SSH private keys accessing our hardware to GitHub PAT tokens for runner registration, each credential type needs its own specific policy. Let's explore how to implement Change Request Policies to secure our automation infrastructure using two distinct policies to handle our different credential types.

Creating change request policies

To establish a clear framework regarding the approval and denial processes associated with updates to confidential information pertaining to the runners projects, I am in the process of developing a dedicated change request policy tailored specifically for this initiative. Through the user interface, I have the capability to formulate a change request policy that allows me to designate myself as the necessary reviewer for any modifications made. This structured approach will ensure that all updates are thoroughly vetted and managed in an organized manner, thereby enhancing the overall integrity and security of the project.

When I find myself in a position where I need to either rotate the secrets or implement a necessary hotfix, I have the capability to initiate a review for myself. This process provides me with a set of guardrails that help prevent any accidental modifications that could lead to potential issues. Additionally, this self-review not only acts as a safety measure but also allows for a thorough audit trail of the changes made.

As my team expands, or in scenarios where I am working within a larger enterprise environment, it becomes increasingly important to include multiple reviewers in the process. These reviewers would ideally come from both the information security engineering team and the credential owner's team, especially when dealing with sensitive changes in a production environment. This collaborative approach ensures that all modifications are carefully scrutinized, enhancing the overall security and integrity of our systems.

Monitoring secret usage with the Analytics Dashboard

While Change Request Policies provide governance over secret modifications, the Analytics Dashboard offers a different but complementary type of insight. Rather than tracking real-time access patterns, this dashboard helps identify stale and potentially unnecessary secrets within your environment.

The Analytics Dashboard is primarily designed to answer questions like:

• Which secrets haven't been updated in extended periods?
• Are there configuration environments that haven't been accessed in weeks or months?
• What secrets might be candidates for cleanup or retirement?

In our GitHub Runner automation context, this becomes valuable for identifying:

• SSH keys that haven't been rotated according to security policies
• Runner tokens that may have been set up but are no longer in active use
• Configuration environments that might have been created for testing but never decommissioned

Rather than providing real-time introspection into access patterns, the dashboard gives us a longer-term view of our secrets hygiene. For example, if we notice the MACMINI3PRIVATEKEY hasn't been updated in over 120 days while all others have been rotated, we can flag it for review. Similarly, if a test environment configuration hasn't been fetched in months, we might consider whether it's still needed.

This long-term visibility helps us maintain a cleaner, more secure secrets ecosystem by:

• Identifying opportunities for secrets cleanup
• Supporting regular rotation practices
• Reducing the overall attack surface by removing unnecessary credentials
• Ensuring our automation relies only on actively maintained configurations

Orchestrating runner deployment with Terraform

With our Mac Minis providing consistent environments via Nix/Home Manager and our secrets securely managed in Doppler, we turn to Terraform on our deployment machine to automate the GitHub Actions runner setup.

Our Terraform configuration will achieve the following for each Mac Mini:

1. Fetch the Mac Mini's specific SSH private key and a fresh GitHub Runner registration token from Doppler.
2. Connect to the Mac Mini via SSH.
3. Execute a script on the Mac Mini to download the GitHub Actions runner binaries, configure the runner using the unique token, ensure it uses the Nix environment, and install it as a system service.

Terraform configuration

We will have two primary Terraform files to define our Mac Mini targets and other parameters, and main.tf will host the core logic. vars.tf defines the structure for our Mac Mini configurations and various global settings.

Now, the main.tf file is too large to show here in this blog, but you can find the full code and everything I am demonstrating on my GitHub page. A quick look will show that we are leveraging both the null_resource, triggers, and the Doppler provider to fetch and update the runners dynamically if we ever need to quickly rotate our runners’ tokens.

Script logic: The sequence of commands (create dir, download, extract, configure, modify ⁠run.sh, install service) is what we troubleshooted and refined. Note the use of single ⁠$ for shell variables (e.g., ⁠$_GITHUB__URL) because they were already assigned their values from Terraform. The ⁠$$ is only needed if there's a risk of Terraform re-interpreting a shell variable placeholder. In this script, we've structured it to avoid that by assigning all Terraform-derived values to shell variables first.

• ⁠--replace flag for `⁠config.sh: This is important. If a runner with the same name already exists (perhaps from a previous failed attempt or if you're re-running with a new token), ⁠–replace will update it.

The role of Doppler's new features in this context

• Doppler Change Requests: While our Terraform script automates the use of the GitHub runner tokens, the process of obtaining these tokens from GitHub and updating them in Doppler is a critical control point. If a team member needs to refresh a token for a Mac Mini runner, Doppler's Change Requests can enforce an approval workflow. The new token is proposed as a change to the ⁠MACMINI\<x\>GHRUNNER_TOKEN secret in Doppler, reviewed (e.g., by a security admin or team lead), and only upon approval does the secret get updated. The next Terraform run for that node will then securely pick up this vetted token, ensuring no unauthorized or accidental runner re-registrations.
• Doppler Analytics Dashboard: Our Terraform process, running on the deployment machine, authenticates to Doppler using a service token (⁠DOPPLER_TOKEN). The Analytics Dashboard in Doppler provides a clear view of this service token's activity. We get insights into stale secrets, such as our machine private keys, and are quickly informed when they should be rotated

This Terraform setup, combined with the described Doppler workflows, provides a robust and auditable way to manage your macOS GitHub Actions runners.

The power of declarative runners and advanced SecretOps: Maintaining your runner fleet

We've journeyed from the complexities of managing macOS CI/CD environments to a streamlined, automated, and highly secure solution. By combining the declarative power of Nix and Home Manager for consistent runner environments, the orchestration capabilities of Terraform, and the advanced secret operations provided by Doppler, we've built a system that is not only efficient but also instills confidence.

The days of snowflake runner configurations and insecurely managed credentials can be a thing of the past. With Terraform fetching SSH keys and ephemeral GitHub runner registration tokens directly from Doppler, our deployment process is both automated and secure. Doppler's role extends beyond simple storage; features like Change Requests introduce crucial governance by allowing approval workflows for updates to sensitive secrets like SSH keys or master API tokens.

Furthermore, the Analytics Dashboard offers invaluable insights into secret access patterns, bolstering our security posture by enabling us to monitor and verify that our automation (and the service tokens it uses) are behaving as expected.

This approach empowers teams to:

• Ensure consistency: Every CI job runs in an identical environment thanks to Nix/Home Manager.
• Automate securely: Terraform and Doppler work in concert to automate setup without exposing secrets.
• Enhance governance: Doppler's Change Requests provide an audit trail and approval layer for critical secret modifications.
• Improve observability: The Analytics Dashboard helps monitor secret usage, adding a proactive security dimension.

Next steps and further considerations

• Scaling: While we focused on a few Mac Minis, this pattern can be scaled. For very large fleets, consider API-driven GitHub runner token generation managed via a secure workflow, with the API's master token itself stored and governed by Doppler.
• Ephemeral Runners: We used the ⁠–ephemeral flag in our example, which is great for security as runners unregister after each job. Consider if this fits all your use cases or if you need persistent runners.
• Monitoring the Runners themselves: Implement monitoring for the Mac Minis (CPU, memory, disk) and the GitHub Actions runner service health.
• Security hardening: Continuously review and harden the security of your Mac Minis, your GitHub organization settings, and your Doppler access controls.
• Explore more Doppler features: Dive deeper into Doppler's capabilities, such as dynamic secrets, secrets referencing, and advanced access controls to further enhance your SecretOps.

Building a secure CI/CD platform is an investment, but the returns in developer productivity, reliability, and security are immense. By adopting declarative tools and a mature SecretOps platform like Doppler, you're well on your way to a best-in-class automation setup.