Table of Contents

    encryption

    SYSTEM DESIGN • CHAPTER 13.5

    Encryption

    Understand how encryption protects data in transit, at rest, and in use, which algorithm families solve which problems, and how key management determines whether any of it actually works.

    Learning objective: By the end of this article, you will understand symmetric and asymmetric encryption, authenticated encryption, hashing versus encryption, transport and storage protection, envelope encryption, key hierarchies, rotation, and the threats each control does and does not address.

    Prerequisites

    Recommended Knowledge

    • Bytes, encoding, and binary data representation
    • HTTP and the role of TLS
    • Databases, object storage, and backups
    • Token signing and verification
    • Trust boundaries from threat modeling
    • Authorization and access control basics
    • Secrets handling in deployed applications
    • Logging and audit fundamentals

    What Encryption Provides

    Encryption transforms readable data into a form that is unintelligible without the correct key. Its purpose is confidentiality: ensuring that data exposed to an unauthorized party remains unusable.

    CORE OPERATION
    \[ C = E_k(P) \qquad P = D_k(C) \]

    Here \(P\) is plaintext, \(C\) is ciphertext, \(k\) is the key, and \(E\) and \(D\) are the encryption and decryption functions. The security of the scheme rests entirely on the secrecy of \(k\), never on the secrecy of the algorithm.

    Simple Analogy

    A safe protects its contents only while the combination stays private. The manufacturer's design being public does not weaken it. Losing control of the combination defeats it entirely, regardless of how strong the steel is.

    KERCKHOFFS'S PRINCIPLE
    A system should remain secure even if everything about it except the key is public knowledge. Secrecy of the algorithm is not a security control.

    What Encryption Does Not Provide

    Encryption is frequently treated as a general-purpose answer to security requirements. It addresses a narrow, specific concern.

    Encryption Addresses

    • Stolen disks, backups, and storage media
    • Interception of network traffic
    • Exposure of misconfigured storage buckets
    • Data readability after infrastructure compromise

    Encryption Does Not Address

    • An authorized user abusing legitimate access
    • Application logic flaws exposing data
    • Broken authorization returning wrong records
    • Compromised credentials used to decrypt normally
    • Data leaked through logs or error messages
    Encrypted storage does not protect against an application that willingly decrypts and returns data to the wrong caller. Authorization remains the primary control.

    Symmetric Encryption

    Symmetric encryption uses one key for both encryption and decryption. It is fast, well suited to bulk data, and forms the basis of nearly all storage and transport protection.

    SYMMETRIC MODEL
    Plaintext Shared Key Ciphertext
    Property Characteristic
    Key count One shared key for both operations
    Speed Fast, suitable for large volumes
    Main challenge Distributing the key securely
    Typical use Database fields, files, backups, session traffic
    Common algorithm AES, typically with 256-bit keys
    The Distribution Problem If two parties must share a key before communicating, how does the key itself travel safely? Symmetric encryption alone cannot solve this, which is precisely why asymmetric cryptography exists.

    Asymmetric Encryption

    Asymmetric encryption uses a mathematically related key pair. The public key may be distributed freely, while the private key remains secret. Data encrypted with one can be decrypted only with the other.

    ASYMMETRIC MODEL
    Public Key Encrypts Private Key Decrypts
    Aspect Symmetric Asymmetric
    Keys involved One shared secret Public and private pair
    Relative speed Fast Considerably slower
    Data volume Large payloads Small payloads such as keys
    Distribution Requires a secure channel Public key needs no secrecy
    Signatures Not applicable Supports digital signatures
    Typical role Encrypting the actual data Key exchange and identity proof
    Practical combination: Real systems rarely choose one. Asymmetric cryptography establishes a shared symmetric key, and symmetric encryption then protects the actual data. TLS works exactly this way.

    Hashing Is Not Encryption

    Hashing produces a fixed-length digest from input of any size and is deliberately irreversible. It answers a different question than encryption and must never be substituted for it.

    Property Encryption Hashing
    Reversible Yes, with the key No, by design
    Output size Related to input size Fixed length
    Requires a key Yes No, though salts are used
    Primary purpose Confidentiality Integrity and verification
    Typical use Protecting readable data Password storage and checksums
    Serious Mistake Storing passwords with encryption rather than hashing. Whoever obtains the key recovers every password in plaintext, which is exactly the outcome password storage must prevent.
    Correct Approach Store passwords using a purpose-built password hashing function with a unique per-user salt and a deliberately high work factor, so verification is possible but recovery is not.

    Authenticated Encryption

    Encryption alone hides content but does not prove that ciphertext arrived unmodified. An attacker may be unable to read the data yet still able to alter it in meaningful ways.

    Authenticated encryption combines confidentiality with integrity, producing an authentication tag that fails verification if anything was tampered with.

    AUTHENTICATED ENCRYPTION
    Plaintext Ciphertext + Auth Tag

    Why This Matters

    • Detects any modification of the ciphertext
    • Prevents silent corruption from being decrypted
    • Binds associated context such as a record identifier
    • Removes the risk of combining primitives incorrectly
    • Is the recommended default for new designs
    Nonce discipline: Authenticated modes require a unique value per encryption operation under a given key. Reusing that value can catastrophically undermine both confidentiality and integrity.

    Encryption in Transit

    Data moving across a network is exposed to interception and modification. Transport Layer Security protects it by establishing an authenticated, encrypted channel.

    TLS ESTABLISHMENT
    Identity Verified Keys Agreed Encrypted Channel
    Guarantee What It Means Threat Prevented
    Confidentiality Traffic is unreadable to observers Passive interception
    Integrity Modification is detected Traffic tampering and injection
    Authentication The server proves its identity Impersonation and interception proxies
    Forward secrecy Past sessions stay safe after key compromise Retrospective decryption of recorded traffic

    Internal Traffic Is Not Exempt

    Outdated Assumption Treating the internal network as trusted and leaving service-to-service traffic unencrypted. Any foothold inside the perimeter then yields readable traffic across the estate.
    Current Practice Encrypt internal traffic as well, with mutual authentication where both parties present certificates, so services verify each other rather than assuming network location implies trust.

    Transport Configuration Essentials

    • Use current protocol versions and disable obsolete ones
    • Validate certificates fully, including the chain and hostname
    • Never disable verification to resolve an error
    • Prefer cipher suites offering forward secrecy
    • Redirect plaintext requests to secure endpoints
    • Monitor certificate expiry well in advance
    • Automate renewal to avoid manual lapses
    • Apply the same standard to internal service calls

    Encryption at Rest

    Stored data may be exposed through stolen media, misconfigured storage, copied backups, or decommissioned hardware. Encryption at rest addresses these physical and configuration exposures.

    Level Scope Protects Against Does Not Protect Against
    Full disk Entire volume Stolen or discarded hardware Anything read through the running system
    Filesystem Directories and files Offline file access Processes with mount access
    Database transparent Data files and logs Stolen database files and backups Any authenticated query
    Column or field Specific sensitive columns Broad database access Callers holding the decryption key
    Application level Values encrypted before storage Database administrators and dumps Compromise of the application itself
    Client side Encrypted before transmission The service provider entirely Server-side search and processing
    THE CRITICAL DISTINCTION
    Transparent database encryption protects files on disk, not rows returned by queries. It defends against stolen media, not against an attacker who can query the database.

    Envelope Encryption

    Envelope encryption solves a practical problem: encrypting large volumes of data while keeping master keys tightly controlled and rotation feasible.

    ENVELOPE PATTERN
    Data Key Encrypts Data Master Key Encrypts Data Key
    1. A unique data key is generated for the item being protected.
    2. The data is encrypted locally using that data key.
    3. The data key is encrypted by a master key held in a key service.
    4. The encrypted data key is stored alongside the ciphertext.
    5. The plaintext data key is discarded from memory.
    6. Decryption reverses the process through the key service.
    {
        "recordId": "customer-4821",
        "ciphertext": "base64-encoded-encrypted-payload",
        "encryptedDataKey": "base64-encoded-wrapped-key",
        "nonce": "base64-encoded-unique-value",
        "authTag": "base64-encoded-authentication-tag",
        "masterKeyId": "key-customer-pii-2026-09",
        "algorithm": "AES-256-GCM",
        "encryptedAt": "2026-09-24T04:12:00Z"
    }

    Advantages

    • Master key never leaves the key service
    • Bulk encryption happens locally and quickly
    • Rotating the master key rewraps only data keys
    • Per-record keys limit blast radius
    • Key usage is centrally audited

    Considerations

    • Key service becomes a critical dependency
    • Wrapped keys add storage overhead
    • Caching data keys creates exposure windows
    • Key service outage blocks decryption

    Key Hierarchy

    A layered key structure limits how much data any single key protects and makes rotation practical.

    Tier Purpose Storage Location Rotation Difficulty
    Root key Protects master keys Hardware security module Rare and carefully planned
    Master key Wraps data keys Managed key service Straightforward, rewrap only
    Data key Encrypts actual records Stored wrapped beside data Requires re-encrypting data
    KEY HIERARCHY
    Root Key Master Key Data Key Encrypted Data

    Key Rotation

    Rotation limits the quantity of data protected by any single key and bounds the damage if a key is exposed. It must be designed in, because retrofitting it is considerably harder.

    CREATE TABLE encryption_keys (
        key_id          VARCHAR(80) PRIMARY KEY,
        key_purpose     VARCHAR(60)  NOT NULL,
        algorithm       VARCHAR(40)  NOT NULL,
        status          VARCHAR(20)  NOT NULL,
        created_at      TIMESTAMP    NOT NULL,
        activated_at    TIMESTAMP    NULL,
        deprecated_at   TIMESTAMP    NULL,
        destroyed_at    TIMESTAMP    NULL
    );
    
    SELECT key_id,
           key_purpose,
           status,
           created_at
    FROM encryption_keys
    WHERE status = 'active'
      AND created_at < CURRENT_TIMESTAMP - INTERVAL '365 days'
    ORDER BY created_at;
    Key State Encrypt Decrypt Meaning
    Pending No No Created but not yet in service
    Active Yes Yes Current key for new operations
    Deprecated No Yes Retained to read existing data
    Disabled No No Suspended, recoverable
    Destroyed No No Unrecoverable, data permanently unreadable
    async function rotateRecordEncryption(recordId) {
        const record = await repository.findEncrypted(recordId);
    
        const plaintext = await cryptoService.decrypt({
            ciphertext: record.ciphertext,
            encryptedDataKey: record.encryptedDataKey,
            nonce: record.nonce,
            authTag: record.authTag,
            masterKeyId: record.masterKeyId
        });
    
        const reEncrypted = await cryptoService.encrypt({
            plaintext: plaintext,
            masterKeyId: await keyService.getActiveKeyId("customer-pii")
        });
    
        await repository.updateEncrypted(recordId, {
            ciphertext: reEncrypted.ciphertext,
            encryptedDataKey: reEncrypted.encryptedDataKey,
            nonce: reEncrypted.nonce,
            authTag: reEncrypted.authTag,
            masterKeyId: reEncrypted.masterKeyId,
            rotatedAt: new Date().toISOString()
        });
    
        plaintext.fill(0);
    }
    Destruction is irreversible: Destroying a key renders every record it protects permanently unreadable, including backups. Verify that no data depends on a key before destroying it.

    Crypto-Shredding

    The irreversibility of key destruction can be used deliberately. Encrypting each tenant or subject with a distinct key means destroying that key effectively erases their data everywhere it was replicated.

    Where It Helps

    • Deletion requests across many replicas
    • Data residing in immutable backups
    • Tenant offboarding and account closure
    • Data spread across derived stores

    Where It Falls Short

    • Plaintext copies already extracted elsewhere
    • Data written into logs or analytics in the clear
    • Shared keys covering multiple subjects
    • Aggregates computed before deletion

    Field-Level Encryption in Practice

    Encrypting specific fields protects them from anyone with broad database access, but it constrains how those fields can be queried.

    CREATE TABLE customers (
        customer_id         BIGINT PRIMARY KEY,
        display_name        VARCHAR(200) NOT NULL,
        email_hash          VARCHAR(64)  NOT NULL,
        email_encrypted     VARBINARY(512) NOT NULL,
        phone_encrypted     VARBINARY(512) NULL,
        tax_id_encrypted    VARBINARY(512) NULL,
        encryption_key_id   VARCHAR(80)  NOT NULL,
        created_at          TIMESTAMP    NOT NULL
    );
    
    CREATE UNIQUE INDEX idx_customers_email_hash
        ON customers (email_hash);

    The separate hash column enables exact-match lookup without decryption. Encrypted values cannot be searched by pattern, sorted meaningfully, or compared by range.

    Operation Feasible on Encrypted Fields Workaround
    Exact match Not directly Deterministic hash column
    Prefix search No Separate searchable derived field
    Range comparison No Store a non-sensitive bucket value
    Sorting No Sort on a non-sensitive attribute
    Aggregation No Precompute during ingestion
    Uniqueness Only with deterministic encoding Hash column with a unique index
    Deterministic Encryption Caution Encrypting identical values to identical ciphertext enables equality lookup, but it also reveals which records share a value, leaking distribution information through frequency analysis.

    Alternatives to Encrypting Everything

    1

    Tokenization

    Replace a sensitive value with a meaningless token, keeping the real value in a separate, tightly controlled vault. Most systems then never handle the sensitive data at all.

    2

    Masking

    Display only a partial value, such as the final digits of an account number, where full visibility is unnecessary for the task.

    3

    Minimization

    Avoid collecting or retaining data that is not genuinely required. Data never stored needs no protection and cannot leak.

    4

    Segmentation

    Confine sensitive data to a small, well-defended service rather than distributing it across every component that touches a workflow.

    The cheapest data to protect is the data you chose not to collect. Minimization frequently outperforms sophisticated cryptography.

    Encryption in Use

    Data must generally be decrypted to be processed, creating a window where plaintext exists in memory. Several approaches narrow that window.

    Approach Principle Practical Limitation
    Confidential computing Processing inside a protected hardware enclave Platform dependent, constrained resources
    Homomorphic encryption Computation directly on ciphertext Substantial performance cost
    Secure multi-party computation Joint computation without sharing inputs Complex coordination and overhead
    Short-lived decryption Decrypt only at the moment of use Requires disciplined memory handling

    Handling Plaintext in Memory

    • Decrypt as late as possible and discard promptly
    • Overwrite buffers rather than relying on garbage collection
    • Exclude sensitive values from logs and traces
    • Prevent plaintext from appearing in error messages
    • Disable core dumps on services handling sensitive data
    • Avoid caching decrypted values without an explicit reason

    Threats and Mitigations

    Threat Description Mitigation
    Hardcoded keys Key committed into source or configuration Managed key service and secret scanning
    Nonce reuse Same nonce used twice under one key Random or counter-based unique generation
    Weak randomness Keys derived from predictable sources Cryptographically secure random generation
    Unauthenticated ciphertext Tampering undetected on decryption Authenticated encryption modes
    Downgrade attack Negotiation forced to a weaker protocol Disable obsolete versions and cipher suites
    Certificate validation disabled Verification bypassed during troubleshooting Enforce validation and fail closed
    Plaintext in logs Sensitive values written before encryption Field redaction and logging review
    Unencrypted backups Protected data copied out in the clear Encrypt backups and test restoration
    Key service compromise Attacker gains decryption capability Least privilege, audit logging, anomaly alerts
    Lost keys Data becomes permanently unreadable Escrow, backups, and tested recovery procedures

    Monitoring and Auditing

    Signals Worth Tracking

    • Decryption request volume by key and by caller
    • Unusual bulk decryption activity
    • Authentication tag verification failures
    • Key service latency and error rate
    • Records still encrypted under deprecated keys
    • Key creation, rotation, and destruction events
    • Certificate expiry proximity
    • Connections negotiating weak protocol versions
    • Access to key management permissions
    • Backup encryption verification results

    Common Design Mistakes

    Weak Design

    • Writing custom cryptographic implementations
    • Encrypting passwords instead of hashing them
    • Storing keys beside the data they protect
    • Using encryption without authentication
    • Reusing nonces across operations
    • Assuming storage encryption stops data leaks
    • Leaving internal traffic unencrypted
    • Deferring rotation until an incident occurs
    • Skipping certificate validation to fix errors

    Strong Design

    • Uses vetted libraries and standard algorithms
    • Hashes passwords with a high work factor
    • Keeps keys in a managed key service
    • Uses authenticated encryption by default
    • Generates a unique nonce per operation
    • Treats authorization as the primary control
    • Encrypts internal and external traffic alike
    • Designs rotation in from the start
    • Validates certificates without exception
    IMPLEMENTATION RULE
    Do not implement cryptographic primitives yourself. Use established libraries and standard modes, because the failure modes of custom implementations are subtle and rarely caught in testing.

    System Design Interview Discussion

    Question What Your Answer Should Cover
    What is encrypted and why? Specific sensitive fields and the threat addressed
    Where do keys live? Key service, hierarchy, and access controls
    How are keys rotated? Key states, rewrapping, and re-encryption strategy
    How do queries work on encrypted fields? Hash columns and the loss of range and sort operations
    What if the key service is down? Caching windows, degradation, and availability planning
    How is deletion handled? Crypto-shredding and its limitations
    Is internal traffic encrypted? Mutual authentication between services
    What does this not protect against? Authorization flaws and authorized misuse

    Implementation Checklist

    Production Checklist

    • Classify data before deciding what to encrypt
    • Use established libraries rather than custom code
    • Choose authenticated encryption modes
    • Generate a unique nonce for every operation
    • Hash passwords with a purpose-built function
    • Store keys in a managed key service
    • Separate key access from data access
    • Adopt envelope encryption for bulk data
    • Record the key identifier with every ciphertext
    • Define key states and rotation procedures
    • Encrypt backups and verify restoration
    • Enforce TLS internally and externally
    • Validate certificates and automate renewal
    • Redact sensitive values from logs and traces
    • Audit every decryption and key management action
    • Test key loss, rotation, and recovery scenarios

    Knowledge Check

    1

    Why combine symmetric and asymmetric encryption?

    Asymmetric cryptography solves key distribution without a pre-shared secret, while symmetric encryption handles bulk data efficiently. Each covers the other's weakness.

    2

    Why hash passwords instead of encrypting them?

    Encryption is reversible, so key compromise exposes every password. Hashing permits verification without ever enabling recovery of the original value.

    3

    What does transparent database encryption protect?

    Data files, logs, and backups on disk. It does not restrict what an authenticated query returns, so it offers no defense against application-level compromise.

    4

    Why use envelope encryption?

    It keeps master keys inside a key service while allowing fast local bulk encryption, and it makes master key rotation a rewrapping operation rather than a full re-encryption.

    5

    Why is authenticated encryption preferred?

    Plain encryption hides content but cannot detect modification. Authenticated modes produce a tag that fails verification if the ciphertext was altered.

    Summary

    Encryption provides confidentiality, transforming readable data into a form that requires a key to recover. Symmetric encryption handles bulk data efficiently, asymmetric encryption solves key distribution and enables signatures, and authenticated encryption adds tamper detection.

    Encryption in transit protects data crossing networks, including internal service calls. Encryption at rest protects stored data against stolen media, misconfigured storage, and copied backups, but not against queries issued through a compromised application.

    Envelope encryption and a layered key hierarchy make large-scale protection practical while keeping master keys inside a key service. Rotation must be designed in, and key destruction is irreversible, which crypto-shredding turns into a deletion mechanism.

    Field-level encryption constrains querying, so hash columns, tokenization, masking, and minimization are often better answers than encrypting every attribute. Throughout, key management determines whether the encryption achieves anything at all.

    Key Takeaway

    Encryption is only as strong as its key management. Use vetted libraries and authenticated modes, keep keys in a managed service separate from the data, design rotation from the beginning, and remember that encryption defends against exposure of storage and traffic, never against an application that decrypts data for the wrong caller.