Secrets management best practices: What to log, alert, and audit

Many teams see monitoring as a task that simply needs to be completed after deployment. They set up logs and alerts, but when audited, those logs do not contain sufficient data to trace secret access, unauthorized attempts, or breaches.

Some teams also set up alerts for every access event, which floods responders with noise. Auditing is often treated as a once-a-year collection of screenshots, leaving incomplete trails and exporting only a few sample logs. If a sysadmin can modify or delete the very trail that tracks their own access, then the monitoring system has no integrity to begin with.

Moreover, most secrets management best-practice guides cover monitoring as a single list item. It’s not enough to say "monitor and audit your secrets." How should teams go about it correctly? What should their logs contain? What should they set alerts for? And how should their audit trails be structured to meet compliance requirements such as SOC 2? These questions often go unanswered, creating security gaps.

This guide examines monitoring as it should be: broken into three operations (logging, alerting, and auditing) to answer these questions. By the end, you will have a clear grasp of what effective monitoring looks like when managing secrets and how to implement it correctly.

What to log

Logs are meant to be more than just records, especially when handling sensitive data. They should serve as evidence of activities and events around secret usage. It's not enough for your logs to hold only the accessed secret name or path for systems that store secrets. Configure them also to store information about the requesting identity (which could be a user account, service account, or API key identifier), timestamp, source IP, environment, and outcome (success or denied). Instead of "secret STRIPE_API_KEY was accessed," logs should show "Secret STRIPE_API_KEY accessed by GitHub-Action-Prod via IP 192.x.x.x at 04:02 UTC." At no point should the secret value itself appear in logs.

When you have a comprehensive log, do not overlook denied access outcomes. While successful access receives a lot of attention, a flood of failed attempts is the first sign of credential stuffing or misconfiguration.

Once your logs capture the right data, the next question is what actually deserves an alert.

What to alert on

Research shows that security operations centers receive numerous alerts, and around 70% of them are false positives or low-priority noise. If your team keeps alerts for every event, it can lead to alert fatigue. Situations that require real attention become overlooked. Alerts are meant to bring attention to activity that is out of the norm and are critical for effective secrets management. To meet CC7.2, you must prove that your system can efficiently distinguish noise from anomalies and alert on them.

Here is what you need to have alerts for:

• Access from an unknown IP: Ensure your system can cross-reference access with known VPN exit nodes or office CIDR blocks. A successful login from a residential IP address in a country where you have no employees is a strong indicator of compromised credentials, even if it occurs on the first attempt.
• Access outside normal hours: Your system should track normal operating hours to spot unusual access times and report on them.
• Three or more failed attempts on the same secret within five minutes: This is a high-confidence signal of brute-force attempts and should not be ignored.
• Out-of-scope access: If a payment-processor-pod suddenly attempts to read hr-management-vault, that’s a strong indicator of compromise and a breakdown in strict access controls.
• Break-glass access: When a root or master credential with no assigned service account is accessed, it should trigger an immediate high-priority page to the security team.

Below is a reference table that summarizes how alert events should be categorized to avoid alert fatigue.

It is important that each alert is configured to route to a named owner to avoid silent failures. Like logs, alerts should carry enough context, such as the secret, identity, environment, and attempted access. Below is an example of what an alert should look like:

With your logs and alerts configured correctly, you should maintain a separate, controlled trail of logs for auditing.

How to audit

To audit correctly, you first need to understand the difference between logs and audit trails. Logs are utilities; they are a raw stream of data for debugging and alerting, while audit trails are meant to prove integrity. You have to maintain an immutable record that demonstrates compliance and aligns with established security practices.

SOC 2 CC7.2 requires that systems be monitored and that security-relevant events can be analyzed over time. In practice, this means your audit trail must be verifiable. If you claim that only your CI/CD runner accessed your production secrets, you must be able to query the trail and show zero human access events for the entire review period. This also means the trail must be retained long enough to support that review.

The minimum practical retention time is 12 months, but different tiers and compliance frameworks have their own requirements. The table below outlines them.

But how do you get audit trails? You build them from logs.

First, implement strict access controls so the principals who access secrets cannot modify the logs that record their access. Otherwise, your audit reviews will fail. Use log forwarders such as Fluent Bit, Datadog Agent, or Vector to collect access data across all of your logs. From these, relevant events are used as audit logs to build audit trails. Importantly, the audit trail should prioritize high-value identity-related events such as authentication, access, denials, and permission changes. Ensure you link these events to provide a complete view of how your systems, cloud providers, and team access and use secrets.

An alternative to building your own is using a dedicated secrets management tool like Doppler. The secrets manager provides an activity log that records secrets access events with the required fields in a queryable format.

While these principles of logging, alerting, and auditing apply broadly, CI/CD pipelines introduce unique risks that require more targeted monitoring.

Secrets monitoring in CI/CD pipelines

CI/CD pipelines are among the most high-traffic transit zones for secrets. A single pipeline run can use SSH keys, database credentials, encryption keys from hardware security modules, and dynamic secrets issued by native automation engines or integrated third-party services. An exposure during this process could lead to unauthorized access to critical data and workflows via stolen credentials.

CI systems like GitHub and GitLab provide auto-masking for secrets, but the masking can fail when the secret is base64-encoded or partially truncated. In GitHub, for example, if secrets are passed as inputs to a workflow dispatcher instead of being registered via ${{ secrets.NAME }}, tools that echo environment variables will log those credentials in plaintext. Even if you delete the log, a secret leaked in a build log has effectively been committed to history. It should be considered compromised, and appropriate secrets rotation should follow. You must explicitly use the ::add-mask:: command.

Make sure you log every pipeline run that accesses secrets. The context of access, such as pipeline name/ID, job or step context, trigger identity (manual or automated), runner IP, and secret scope, would provide evidence of who/what used your secrets, how, and why.

Lastly, implement anomaly detection for secrets usage within your pipelines. Configure your system to alert you if pipelines access secrets outside their defined scope and when a CI/CD runner’s IP address falls outside your known CIDR blocks.

When you have effectively set up secrets monitoring across your secrets stores, pipelines, and external systems, it’s important to be ready when anomalies surface.

When monitoring reveals a problem

The point of monitoring is to enable an efficient incident response as part of how teams implement secrets management. The best way to think about it is as the destination for your monitoring data. When monitoring surfaces a real incident, you must have an audit-ready response that resolves it promptly without breaking the evidence chain.

To achieve this, define the response for each alert class before an incident occurs. Responses have to be governed by documented processes, or auditors may question the reliability of your monitoring and response controls. Every high-confidence alert class should be mapped to a specific playbook.

For an impossible travel alert, you might first scope the blast radius to determine which systems have access to the secret and what was accessed before mass revocation. Revoking access before mapping dependencies can cause you to miss part of a breach or introduce outages. For example, you might revoke a Stripe API token due to a breach in the payment microservice, only for checkout to fail as a result.

You should also log the incident investigation and response itself. The record must include who saw the alert and when they saw it, which logs were reviewed, whether the event was a false positive or a confirmed breach, and what action was taken. This record should be part of your audit trail.

Effective secrets monitoring comes from connecting logs, alerts, and audits into a response process that can detect incidents, preserve evidence, and support rapid remediation when secrets are exposed.

The best secrets management strategy is one your team will actually use. Doppler helps you centralize secrets, automate rotation, enforce least-privilege access, and securely deliver credentials to every environment without slowing developers down. See how Doppler can help you implement these best practices here.