Where Are KMS Encryption Keys Stored? - How Envelope Encryption and HSMs Work
This article explains how KMS master keys are stored inside FIPS 140-2 certified HSMs, how envelope encryption provides double protection for data keys, and the internal workings of key rotation.
KMS Keys Are Never Stored in Plaintext
Encryption keys (KMS keys, formerly called CMKs) created in AWS KMS (Key Management Service) are never stored in plaintext on disk. The key material (the actual cryptographic key data) of a KMS key exists in plaintext only inside HSMs (Hardware Security Modules) certified to FIPS 140-2 Level 3. An HSM is a dedicated hardware device for cryptographic operations, equipped with physical tamper resistance (the ability to detect unauthorized opening or modification and automatically erase keys). When a KMS key is stored outside the HSM, it is stored in encrypted form (ciphertext), encrypted by a domain key internal to the HSM. When encryption or decryption is needed, the encrypted key material is loaded into the HSM, decrypted inside the HSM for cryptographic operations, and erased from HSM memory once processing is complete. In other words, what is stored in AWS's storage systems is encrypted key material, and only the HSM can decrypt it. Even if an AWS employee accesses the storage, they can only see encrypted key material.
Envelope Encryption - Why Encrypt Keys with Keys?
The core of KMS's encryption model is "envelope encryption." Rather than encrypting data directly with a KMS key, you first request KMS to generate a data key, encrypt the data with that data key, and then encrypt the data key itself with the KMS key for storage. Why is this two-layer structure necessary? There are two reasons. First, performance. The KMS encryption API can process only up to 4KB of data per request. Encrypting a multi-gigabyte file would require millions of requests to KMS, which is impractical. Using a data key locally allows you to encrypt data of any size at high speed. Second, latency. KMS API calls incur network round-trip time. Caching the data key in memory eliminates the need to call KMS for every encryption operation. S3's SSE-KMS (Server-Side Encryption with KMS) automatically performs this envelope encryption internally. A unique data key is generated for each object, the object's data is encrypted with the data key, and the data key is encrypted with the KMS key and stored as object metadata.
Internal Workings of Key Rotation
When you enable automatic key rotation for KMS, new key material is generated annually (or every 90 to 2,560 days depending on configuration). The important point here is that old key material is not deleted after key rotation. The KMS key's ARN and Key ID remain unchanged, data encrypted with old key material is automatically decrypted using the old key material, and new data encryption uses the new key material. This design makes key rotation completely transparent. No application code changes are needed, and re-encryption of existing data is not required. Internally, KMS embeds information in the ciphertext indicating which version of key material was used for encryption, and automatically selects the appropriate key material during decryption. With manual key rotation (creating a new KMS key and switching the alias), you need to re-encrypt data encrypted with the old KMS key using the new KMS key. If you don't understand the difference between automatic and manual rotation, you may encounter incidents where old data becomes undecryptable after manual rotation.
Choosing Between KMS and CloudHSM
KMS is a managed service that operates on multi-tenant HSMs. Multiple customers' KMS keys are processed on the same HSM cluster, but each key is cryptographically isolated, making it impossible to access another customer's keys. CloudHSM, on the other hand, provides single-tenant HSMs. Dedicated HSM appliances are placed within your VPC, accessible only to you. Cases where CloudHSM is needed are limited. First, when regulatory requirements demand "dedicated HSMs." Some regulations in financial institutions and government agencies require managing encryption keys on dedicated hardware. Second, when you need to use standard cryptographic APIs such as PKCS#11, JCE, or CNG. KMS uses AWS-proprietary APIs, while CloudHSM supports industry-standard cryptographic APIs. Third, when you need cryptographic processing that exceeds KMS's request rate limits (default of 5,500 to 30,000 requests per second, varying by region). CloudHSM has no rate limits since it runs on dedicated hardware. For most use cases, KMS is sufficient, and CloudHSM should only be considered when special requirements exist.
Key Deletion - The Most Dangerous Irreversible Operation
Deleting a KMS key is one of the most dangerous operations in AWS. When you delete a KMS key, all data encrypted with that key becomes permanently undecryptable. S3 objects, EBS volumes, RDS snapshots - all data dependent on that key is effectively lost. To mitigate this risk, AWS imposes a 7-to-30-day waiting period for KMS key deletion. When you schedule deletion, the key enters a "Pending Deletion" state and cannot be used for encryption or decryption during the waiting period. If you cancel the deletion during the waiting period, the key is restored. Once the waiting period expires, the key material is completely erased from the HSM and can never be recovered. If a key in pending deletion state is used (an encryption or decryption request occurs), it is logged in CloudTrail. Monitoring these logs lets you verify whether resources still depend on that key. For production KMS keys, it is strongly recommended to restrict the kms:ScheduleKeyDeletion action in the key policy to prevent accidental deletion. For a systematic study of encryption and key management design, specialized books on Amazon are a helpful reference.