KMS · By · 3 min read

Using AWS KMS for secure SSH credential management

Storing SSH credentials safely is harder than it looks. AWS KMS fits into a modern access flow in specific ways, with specific frictions and pitfalls worth naming.

AWS KMS is an excellent piece of infrastructure. It is also frequently misused in SSH credential flows in ways that make incident response harder, not easier. The way we think about it is shaped by watching a handful of teams get bitten by the same mistakes.

What KMS actually gives you

KMS stores a master key you never see, encrypts/decrypts payloads under it, and logs every use. For SSH, the typical pattern is:

  1. Generate an SSH keypair in your CI or control plane.
  2. Encrypt the private key with KMS.
  3. Store the ciphertext in S3, Secrets Manager, or a database.
  4. At use time, decrypt the ciphertext with a KMS call.

This gives you: audit logs of who touched the key, centralized access control via IAM, and separation between “can see the encrypted blob” and “can decrypt it.”

Where it gets misused

Long-lived decrypted keys. Teams decrypt the key at app startup and hold it in memory for the process lifetime. If the app is compromised, so is the key. Decrypt per-use and zeroize.

Over-broad KMS grants. A single IAM role that can decrypt every SSH key in the org is the opposite of least privilege. Scope decrypt permissions per environment, and use KMS grants instead of IAM policies for fine-grained control.

Missing audit trails. CloudTrail logs every KMS Decrypt call, but only if you have a multi-region trail enabled. Many teams find this out during an incident.

The pattern that works

For SSH specifically, we use KMS to wrap session-scoped SSH certificates, not long-lived keys. The flow:

  1. Operator authenticates via SSO.
  2. Control plane mints a short-lived SSH cert signed by a CA private key.
  3. The CA private key lives in KMS; every signing operation is a Sign call, the key itself never exits KMS.
  4. The cert is delivered to the agent for the specific session.
  5. Cert expires in 15 minutes; no rotation needed.

Two properties fall out of this:

  • The most sensitive secret (the CA private key) never leaves KMS.
  • Every operator session shows up as a KMS Sign event you can correlate to an operator identity.

When not to use KMS

If you don’t need audit or cross-account key sharing, a local KMS is overkill. For a single-region, single-account setup, Secrets Manager with rotation is simpler and serves most teams.

KMS is a force multiplier when you already have a multi-account, audit-heavy environment. It’s not a magic box that makes SSH safer, it’s a durable place to put the single keystone secret in a credential-minting flow.

Related posts