On January 4th, CircleCI posted a security notice alluding to a breach of customer API tokens, and consequently, secrets. Further details, such as what exactly happened, aren’t public yet. But, the notice mentions multiple times that secrets should be immediately rotated. If it were only that easy.
☣️ Responding to breaches and leaks will continue to be hard until services support programmatically managing secrets
Secrets (and its sibling passwords) aren’t perfect; they are, however, ubiquitous. The majority of organizations, at one time or another, will leak some or all of their secrets. Without an effective mitigation and response strategy, you’re stuck using a spreadsheet to wade through what needs to be rotated, hoping you don’t miss anything. The entire time your CTO is breathing down your neck because the CEO is breathing down theirs. Most importantly, your customers’ lives may be materially impacted.
Secrets rotation has always been encouraged but remained a TODO. Reasons for not performing rotation often included potential downtime and fear of the unknown. The root of these issues - orchestration and observability - have largely been addressed.
There is one reason, however, that still persists and is the single biggest inhibitor to performing rotation - programmatic support for managing secrets in third-party tools.
☣️ Requiring users to login to a dashboard to manage secrets is not scalable. There are few things users would rather be doing less when responding to an incident
It may seem intimidating to make managing secrets more accessible, but it’s actually more secure.
There are only a handful of endpoints needed to support managing secrets programmatically (preferably via OAuth):
Once these endpoints are implemented you'll just need to ensure you support more than one active secret instance at a time. That’s it.
Doppler set out to help organizations become more secure when using secrets. We’ve spent a lot of time thinking about rotation. A lot. We’re building towards a world where secrets are regularly rotated - automatically; where leaks can be responded to with the click of a button; where rotation doesn’t involve downtime and deployments are triggered automatically.
If you are a CircleCI customer leveraging Doppler secrets rotation, you can rotate five of the most commonly used secrets - instantly. And we’ve got a lot more on the way.
We’re excited about making companies more secure through rotation. You can check out a detailed write-up here. We think engineers will appreciate the decisions we’ve made and the outcomes we support.
CircleCI’s notice triggered this blog post, but it’s not a dig at them. The intent is to highlight how close we actually are to making our industry fundamentally more secure. One may belittle secrets all they’d like, but we prefer to acknowledge things as they are and then make them better.
Doppler Co-Founder & CTO