encryption
Encryption
Understand how encryption protects data in transit, at rest, and in use, which algorithm families solve which problems, and how key management determines whether any of it actually works.
Prerequisites
Recommended Knowledge
- Bytes, encoding, and binary data representation
- HTTP and the role of TLS
- Databases, object storage, and backups
- Token signing and verification
- Trust boundaries from threat modeling
- Authorization and access control basics
- Secrets handling in deployed applications
- Logging and audit fundamentals
What Encryption Provides
Encryption transforms readable data into a form that is unintelligible without the correct key. Its purpose is confidentiality: ensuring that data exposed to an unauthorized party remains unusable.
Here \(P\) is plaintext, \(C\) is ciphertext, \(k\) is the key, and \(E\) and \(D\) are the encryption and decryption functions. The security of the scheme rests entirely on the secrecy of \(k\), never on the secrecy of the algorithm.
Simple Analogy
A safe protects its contents only while the combination stays private. The manufacturer's design being public does not weaken it. Losing control of the combination defeats it entirely, regardless of how strong the steel is.
What Encryption Does Not Provide
Encryption is frequently treated as a general-purpose answer to security requirements. It addresses a narrow, specific concern.
Encryption Addresses
- Stolen disks, backups, and storage media
- Interception of network traffic
- Exposure of misconfigured storage buckets
- Data readability after infrastructure compromise
Encryption Does Not Address
- An authorized user abusing legitimate access
- Application logic flaws exposing data
- Broken authorization returning wrong records
- Compromised credentials used to decrypt normally
- Data leaked through logs or error messages
Symmetric Encryption
Symmetric encryption uses one key for both encryption and decryption. It is fast, well suited to bulk data, and forms the basis of nearly all storage and transport protection.
| Property | Characteristic |
|---|---|
| Key count | One shared key for both operations |
| Speed | Fast, suitable for large volumes |
| Main challenge | Distributing the key securely |
| Typical use | Database fields, files, backups, session traffic |
| Common algorithm | AES, typically with 256-bit keys |
Asymmetric Encryption
Asymmetric encryption uses a mathematically related key pair. The public key may be distributed freely, while the private key remains secret. Data encrypted with one can be decrypted only with the other.
| Aspect | Symmetric | Asymmetric |
|---|---|---|
| Keys involved | One shared secret | Public and private pair |
| Relative speed | Fast | Considerably slower |
| Data volume | Large payloads | Small payloads such as keys |
| Distribution | Requires a secure channel | Public key needs no secrecy |
| Signatures | Not applicable | Supports digital signatures |
| Typical role | Encrypting the actual data | Key exchange and identity proof |
Hashing Is Not Encryption
Hashing produces a fixed-length digest from input of any size and is deliberately irreversible. It answers a different question than encryption and must never be substituted for it.
| Property | Encryption | Hashing |
|---|---|---|
| Reversible | Yes, with the key | No, by design |
| Output size | Related to input size | Fixed length |
| Requires a key | Yes | No, though salts are used |
| Primary purpose | Confidentiality | Integrity and verification |
| Typical use | Protecting readable data | Password storage and checksums |
Authenticated Encryption
Encryption alone hides content but does not prove that ciphertext arrived unmodified. An attacker may be unable to read the data yet still able to alter it in meaningful ways.
Authenticated encryption combines confidentiality with integrity, producing an authentication tag that fails verification if anything was tampered with.
Why This Matters
- Detects any modification of the ciphertext
- Prevents silent corruption from being decrypted
- Binds associated context such as a record identifier
- Removes the risk of combining primitives incorrectly
- Is the recommended default for new designs
Encryption in Transit
Data moving across a network is exposed to interception and modification. Transport Layer Security protects it by establishing an authenticated, encrypted channel.
| Guarantee | What It Means | Threat Prevented |
|---|---|---|
| Confidentiality | Traffic is unreadable to observers | Passive interception |
| Integrity | Modification is detected | Traffic tampering and injection |
| Authentication | The server proves its identity | Impersonation and interception proxies |
| Forward secrecy | Past sessions stay safe after key compromise | Retrospective decryption of recorded traffic |
Internal Traffic Is Not Exempt
Transport Configuration Essentials
- Use current protocol versions and disable obsolete ones
- Validate certificates fully, including the chain and hostname
- Never disable verification to resolve an error
- Prefer cipher suites offering forward secrecy
- Redirect plaintext requests to secure endpoints
- Monitor certificate expiry well in advance
- Automate renewal to avoid manual lapses
- Apply the same standard to internal service calls
Encryption at Rest
Stored data may be exposed through stolen media, misconfigured storage, copied backups, or decommissioned hardware. Encryption at rest addresses these physical and configuration exposures.
| Level | Scope | Protects Against | Does Not Protect Against |
|---|---|---|---|
| Full disk | Entire volume | Stolen or discarded hardware | Anything read through the running system |
| Filesystem | Directories and files | Offline file access | Processes with mount access |
| Database transparent | Data files and logs | Stolen database files and backups | Any authenticated query |
| Column or field | Specific sensitive columns | Broad database access | Callers holding the decryption key |
| Application level | Values encrypted before storage | Database administrators and dumps | Compromise of the application itself |
| Client side | Encrypted before transmission | The service provider entirely | Server-side search and processing |
Envelope Encryption
Envelope encryption solves a practical problem: encrypting large volumes of data while keeping master keys tightly controlled and rotation feasible.
- A unique data key is generated for the item being protected.
- The data is encrypted locally using that data key.
- The data key is encrypted by a master key held in a key service.
- The encrypted data key is stored alongside the ciphertext.
- The plaintext data key is discarded from memory.
- Decryption reverses the process through the key service.
{
"recordId": "customer-4821",
"ciphertext": "base64-encoded-encrypted-payload",
"encryptedDataKey": "base64-encoded-wrapped-key",
"nonce": "base64-encoded-unique-value",
"authTag": "base64-encoded-authentication-tag",
"masterKeyId": "key-customer-pii-2026-09",
"algorithm": "AES-256-GCM",
"encryptedAt": "2026-09-24T04:12:00Z"
}
Advantages
- Master key never leaves the key service
- Bulk encryption happens locally and quickly
- Rotating the master key rewraps only data keys
- Per-record keys limit blast radius
- Key usage is centrally audited
Considerations
- Key service becomes a critical dependency
- Wrapped keys add storage overhead
- Caching data keys creates exposure windows
- Key service outage blocks decryption
Key Hierarchy
A layered key structure limits how much data any single key protects and makes rotation practical.
| Tier | Purpose | Storage Location | Rotation Difficulty |
|---|---|---|---|
| Root key | Protects master keys | Hardware security module | Rare and carefully planned |
| Master key | Wraps data keys | Managed key service | Straightforward, rewrap only |
| Data key | Encrypts actual records | Stored wrapped beside data | Requires re-encrypting data |
Key Rotation
Rotation limits the quantity of data protected by any single key and bounds the damage if a key is exposed. It must be designed in, because retrofitting it is considerably harder.
CREATE TABLE encryption_keys (
key_id VARCHAR(80) PRIMARY KEY,
key_purpose VARCHAR(60) NOT NULL,
algorithm VARCHAR(40) NOT NULL,
status VARCHAR(20) NOT NULL,
created_at TIMESTAMP NOT NULL,
activated_at TIMESTAMP NULL,
deprecated_at TIMESTAMP NULL,
destroyed_at TIMESTAMP NULL
);
SELECT key_id,
key_purpose,
status,
created_at
FROM encryption_keys
WHERE status = 'active'
AND created_at < CURRENT_TIMESTAMP - INTERVAL '365 days'
ORDER BY created_at;
| Key State | Encrypt | Decrypt | Meaning |
|---|---|---|---|
| Pending | No | No | Created but not yet in service |
| Active | Yes | Yes | Current key for new operations |
| Deprecated | No | Yes | Retained to read existing data |
| Disabled | No | No | Suspended, recoverable |
| Destroyed | No | No | Unrecoverable, data permanently unreadable |
async function rotateRecordEncryption(recordId) {
const record = await repository.findEncrypted(recordId);
const plaintext = await cryptoService.decrypt({
ciphertext: record.ciphertext,
encryptedDataKey: record.encryptedDataKey,
nonce: record.nonce,
authTag: record.authTag,
masterKeyId: record.masterKeyId
});
const reEncrypted = await cryptoService.encrypt({
plaintext: plaintext,
masterKeyId: await keyService.getActiveKeyId("customer-pii")
});
await repository.updateEncrypted(recordId, {
ciphertext: reEncrypted.ciphertext,
encryptedDataKey: reEncrypted.encryptedDataKey,
nonce: reEncrypted.nonce,
authTag: reEncrypted.authTag,
masterKeyId: reEncrypted.masterKeyId,
rotatedAt: new Date().toISOString()
});
plaintext.fill(0);
}
Crypto-Shredding
The irreversibility of key destruction can be used deliberately. Encrypting each tenant or subject with a distinct key means destroying that key effectively erases their data everywhere it was replicated.
Where It Helps
- Deletion requests across many replicas
- Data residing in immutable backups
- Tenant offboarding and account closure
- Data spread across derived stores
Where It Falls Short
- Plaintext copies already extracted elsewhere
- Data written into logs or analytics in the clear
- Shared keys covering multiple subjects
- Aggregates computed before deletion
Field-Level Encryption in Practice
Encrypting specific fields protects them from anyone with broad database access, but it constrains how those fields can be queried.
CREATE TABLE customers (
customer_id BIGINT PRIMARY KEY,
display_name VARCHAR(200) NOT NULL,
email_hash VARCHAR(64) NOT NULL,
email_encrypted VARBINARY(512) NOT NULL,
phone_encrypted VARBINARY(512) NULL,
tax_id_encrypted VARBINARY(512) NULL,
encryption_key_id VARCHAR(80) NOT NULL,
created_at TIMESTAMP NOT NULL
);
CREATE UNIQUE INDEX idx_customers_email_hash
ON customers (email_hash);
The separate hash column enables exact-match lookup without decryption. Encrypted values cannot be searched by pattern, sorted meaningfully, or compared by range.
| Operation | Feasible on Encrypted Fields | Workaround |
|---|---|---|
| Exact match | Not directly | Deterministic hash column |
| Prefix search | No | Separate searchable derived field |
| Range comparison | No | Store a non-sensitive bucket value |
| Sorting | No | Sort on a non-sensitive attribute |
| Aggregation | No | Precompute during ingestion |
| Uniqueness | Only with deterministic encoding | Hash column with a unique index |
Alternatives to Encrypting Everything
Tokenization
Replace a sensitive value with a meaningless token, keeping the real value in a separate, tightly controlled vault. Most systems then never handle the sensitive data at all.
Masking
Display only a partial value, such as the final digits of an account number, where full visibility is unnecessary for the task.
Minimization
Avoid collecting or retaining data that is not genuinely required. Data never stored needs no protection and cannot leak.
Segmentation
Confine sensitive data to a small, well-defended service rather than distributing it across every component that touches a workflow.
Encryption in Use
Data must generally be decrypted to be processed, creating a window where plaintext exists in memory. Several approaches narrow that window.
| Approach | Principle | Practical Limitation |
|---|---|---|
| Confidential computing | Processing inside a protected hardware enclave | Platform dependent, constrained resources |
| Homomorphic encryption | Computation directly on ciphertext | Substantial performance cost |
| Secure multi-party computation | Joint computation without sharing inputs | Complex coordination and overhead |
| Short-lived decryption | Decrypt only at the moment of use | Requires disciplined memory handling |
Handling Plaintext in Memory
- Decrypt as late as possible and discard promptly
- Overwrite buffers rather than relying on garbage collection
- Exclude sensitive values from logs and traces
- Prevent plaintext from appearing in error messages
- Disable core dumps on services handling sensitive data
- Avoid caching decrypted values without an explicit reason
Threats and Mitigations
| Threat | Description | Mitigation |
|---|---|---|
| Hardcoded keys | Key committed into source or configuration | Managed key service and secret scanning |
| Nonce reuse | Same nonce used twice under one key | Random or counter-based unique generation |
| Weak randomness | Keys derived from predictable sources | Cryptographically secure random generation |
| Unauthenticated ciphertext | Tampering undetected on decryption | Authenticated encryption modes |
| Downgrade attack | Negotiation forced to a weaker protocol | Disable obsolete versions and cipher suites |
| Certificate validation disabled | Verification bypassed during troubleshooting | Enforce validation and fail closed |
| Plaintext in logs | Sensitive values written before encryption | Field redaction and logging review |
| Unencrypted backups | Protected data copied out in the clear | Encrypt backups and test restoration |
| Key service compromise | Attacker gains decryption capability | Least privilege, audit logging, anomaly alerts |
| Lost keys | Data becomes permanently unreadable | Escrow, backups, and tested recovery procedures |
Monitoring and Auditing
Signals Worth Tracking
- Decryption request volume by key and by caller
- Unusual bulk decryption activity
- Authentication tag verification failures
- Key service latency and error rate
- Records still encrypted under deprecated keys
- Key creation, rotation, and destruction events
- Certificate expiry proximity
- Connections negotiating weak protocol versions
- Access to key management permissions
- Backup encryption verification results
Common Design Mistakes
Weak Design
- Writing custom cryptographic implementations
- Encrypting passwords instead of hashing them
- Storing keys beside the data they protect
- Using encryption without authentication
- Reusing nonces across operations
- Assuming storage encryption stops data leaks
- Leaving internal traffic unencrypted
- Deferring rotation until an incident occurs
- Skipping certificate validation to fix errors
Strong Design
- Uses vetted libraries and standard algorithms
- Hashes passwords with a high work factor
- Keeps keys in a managed key service
- Uses authenticated encryption by default
- Generates a unique nonce per operation
- Treats authorization as the primary control
- Encrypts internal and external traffic alike
- Designs rotation in from the start
- Validates certificates without exception
System Design Interview Discussion
| Question | What Your Answer Should Cover |
|---|---|
| What is encrypted and why? | Specific sensitive fields and the threat addressed |
| Where do keys live? | Key service, hierarchy, and access controls |
| How are keys rotated? | Key states, rewrapping, and re-encryption strategy |
| How do queries work on encrypted fields? | Hash columns and the loss of range and sort operations |
| What if the key service is down? | Caching windows, degradation, and availability planning |
| How is deletion handled? | Crypto-shredding and its limitations |
| Is internal traffic encrypted? | Mutual authentication between services |
| What does this not protect against? | Authorization flaws and authorized misuse |
Implementation Checklist
Production Checklist
- Classify data before deciding what to encrypt
- Use established libraries rather than custom code
- Choose authenticated encryption modes
- Generate a unique nonce for every operation
- Hash passwords with a purpose-built function
- Store keys in a managed key service
- Separate key access from data access
- Adopt envelope encryption for bulk data
- Record the key identifier with every ciphertext
- Define key states and rotation procedures
- Encrypt backups and verify restoration
- Enforce TLS internally and externally
- Validate certificates and automate renewal
- Redact sensitive values from logs and traces
- Audit every decryption and key management action
- Test key loss, rotation, and recovery scenarios
Knowledge Check
Why combine symmetric and asymmetric encryption?
Asymmetric cryptography solves key distribution without a pre-shared secret, while symmetric encryption handles bulk data efficiently. Each covers the other's weakness.
Why hash passwords instead of encrypting them?
Encryption is reversible, so key compromise exposes every password. Hashing permits verification without ever enabling recovery of the original value.
What does transparent database encryption protect?
Data files, logs, and backups on disk. It does not restrict what an authenticated query returns, so it offers no defense against application-level compromise.
Why use envelope encryption?
It keeps master keys inside a key service while allowing fast local bulk encryption, and it makes master key rotation a rewrapping operation rather than a full re-encryption.
Why is authenticated encryption preferred?
Plain encryption hides content but cannot detect modification. Authenticated modes produce a tag that fails verification if the ciphertext was altered.
Summary
Encryption provides confidentiality, transforming readable data into a form that requires a key to recover. Symmetric encryption handles bulk data efficiently, asymmetric encryption solves key distribution and enables signatures, and authenticated encryption adds tamper detection.
Encryption in transit protects data crossing networks, including internal service calls. Encryption at rest protects stored data against stolen media, misconfigured storage, and copied backups, but not against queries issued through a compromised application.
Envelope encryption and a layered key hierarchy make large-scale protection practical while keeping master keys inside a key service. Rotation must be designed in, and key destruction is irreversible, which crypto-shredding turns into a deletion mechanism.
Field-level encryption constrains querying, so hash columns, tokenization, masking, and minimization are often better answers than encrypting every attribute. Throughout, key management determines whether the encryption achieves anything at all.
Key Takeaway
Encryption is only as strong as its key management. Use vetted libraries and authenticated modes, keep keys in a managed service separate from the data, design rotation from the beginning, and remember that encryption defends against exposure of storage and traffic, never against an application that decrypts data for the wrong caller.