Introduction

Storing data successfully is not enough. A reliable storage system must preserve that data after the client receives confirmation that the write has completed.

Durability describes the system's ability to keep acknowledged data intact and recoverable according to its documented failure model.

Durability concerns include:

• Process crashes
• Operating-system failures
• Storage-device failures
• Power interruptions
• Network partitions
• Data corruption
• Availability-zone failures
• Regional disasters
• Accidental deletion
• Malicious modification
• Software defects

In your System Design curriculum, Durability is Topic 6.2 under Storage, Files, Objects & Search Basics. It follows block, file, and object storage and precedes checksums, metadata, uploads and multipart transfer, inverted indexes, and full-text search.

Prerequisites

What Is Durability?

Durability is the guarantee that, after a storage system acknowledges a successful write, the stored data remains protected and recoverable under the failures covered by that system's durability contract.

Client sends write
      |
      v
Storage system receives data
      |
      v
Required persistence or redundancy completes
      |
      v
System acknowledges success
      |
      v
Covered failure occurs
      |
      v
Acknowledged data remains present
or can be recovered

Durability is not an unlimited promise. A single-device system, replicated storage service, versioned object store, and geographically distributed backup system protect against different failure scopes.

Durability vs Availability

Durability and availability answer different questions.

Durable but temporarily unavailable:

The data remains safely stored,
but a network outage prevents access.


Available but insufficiently durable:

The data is currently readable,
but only one vulnerable copy exists.

Durability vs Consistency

A replica can durably retain an older value while another replica stores a newer value. Durability alone does not define which version a reader will observe.

Define the Failure Model

A durability guarantee should identify the failures it is designed to tolerate.

Write Acknowledgment

The moment at which the system confirms a write is central to its durability guarantee.

Weak acknowledgment:

1. Receive write.
2. Keep data only in volatile memory.
3. Return success.
4. Process loses power.
5. Acknowledged data can be lost.


Stronger acknowledgment:

1. Receive write.
2. Persist required recovery information.
3. Create required redundant representation.
4. Return success.

Stronger acknowledgment can require additional storage operations or network round trips, increasing write latency.

Volatile vs Persistent Storage

Caches improve performance but should not be treated as the only durable copy unless the cache is explicitly designed and configured as a durable storage system.

Write-ahead Logging

A database can use write-ahead logging so that recovery information is made durable before modified data pages are written to their final locations.

Transaction modifies data
      |
      v
Create transaction-log records
      |
      v
Persist required log records
      |
      v
Acknowledge commit
      |
      v
Write modified data pages later

After a crash, the database can use its transaction log to restore committed operations and remove incomplete work according to its recovery algorithm.

Conceptual Database Transaction

BEGIN;

INSERT INTO enrollments
(
    enrollment_id,
    learner_id,
    course_id,
    enrollment_status
)
VALUES
(
    :enrollment_id,
    :learner_id,
    :course_id,
    'active'
);

COMMIT;

The database durability configuration determines what persistence work must complete before the commit is acknowledged.

Replication

Data replication maintains multiple copies of stored data, called replicas, across selected storage components or locations.

Client write
    |
    v
Primary storage
    |
    +-- Replica A
    |
    +-- Replica B
    |
    +-- Replica C

Replication can help protect against drive, server, rack, zone, or regional failure depending on where replicas are placed.

Synchronous Replication

With synchronous replication, the write acknowledgment waits for the required replica confirmations.

Client sends write
      |
      v
Primary stores write
      |
      v
Required replica stores write
      |
      v
Replica acknowledges
      |
      v
Primary returns success

Potential Benefit

The acknowledged write is already represented on the required participating replicas.

Consideration

Write latency and availability can depend on the slowest required replica or network path.

Asynchronous Replication

With asynchronous replication, the primary can acknowledge a write before a remote replica receives it.

Client sends write
      |
      v
Primary stores write
      |
      v
Primary returns success
      |
      v
Replication continues asynchronously

If the primary is lost before replication completes, recently acknowledged data can be absent from the remote replica, depending on the service guarantee.

Replication Comparison

Quorum-based Persistence

Distributed storage can acknowledge a write after a required subset of replicas confirms it.

Replication factor:

N


Required write acknowledgments:

W


Required read responses:

R

A simplified condition often discussed for overlapping read and write quorums is:

\[ R + W > N \]

This equation alone does not fully describe the consistency or durability of a real system. Replica failures, conflict resolution, repair, timing, and implementation details also matter.

Erasure Coding

Erasure coding divides data into fragments and generates additional parity fragments. The original content can be reconstructed when enough fragments remain available.

Original data
      |
      v
Split into data fragments
      |
      v
Generate parity fragments
      |
      v
Distribute fragments across devices


After selected fragment loss:

Remaining fragments
      |
      v
Reconstruct original content

Erasure coding can provide storage-efficient redundancy but requires encoding, distribution, reconstruction, and repair work.

Replication vs Erasure Coding

Checksums and Integrity

Durability must protect not only against missing data but also against undetected corruption.

Write data
    |
    v
Calculate checksum
    |
    v
Store data and checksum
    |
    v
Read data later
    |
    v
Recalculate checksum
    |
    +-- Match:
    |      content passed verification
    |
    +-- Mismatch:
           corruption detected

When redundant data exists, the system can use a valid copy or fragment set to repair corrupted content.

File-checksum Example

sha256sum course-video.mp4

Checksums and integrity verification are covered in detail in the next topic after durability.

Scrubbing

Data scrubbing is the periodic process of reading stored data, verifying its integrity, and repairing detected corruption when valid redundant data is available.

Periodic verification job
        |
        v
Read stored fragment or object
        |
        v
Verify checksum
        |
        +-- Valid:
        |      continue
        |
        +-- Invalid:
               locate healthy copy
               repair corrupted data
               record event

Without periodic verification, silent corruption can remain undiscovered until recovery is attempted.

Snapshots

A snapshot captures storage state at a specific point in time.

Live volume
    |
    +-- Snapshot at Time A
    |
    +-- Snapshot at Time B
    |
    +-- Snapshot at Time C

Snapshots can support:

• Recovery from accidental modification
• Volume cloning
• Testing against a captured state
• Backup workflows
• Point-in-time recovery strategies

A snapshot stored in the same failure domain can share risks with the source storage. Its independence and recovery value must be verified.

Object Versioning

Object versioning retains previous generations when an object is overwritten or deleted according to the configured policy.

Object key:

courses/42/lesson-7/notes.pdf


Version 1:
Original content


Version 2:
Corrected content


Version 3:
Latest content

Versioning can help recover from accidental overwrite or deletion, but it increases storage consumption and requires lifecycle and retention rules.

Backups

A backup is a timestamped copy of data that can be used to restore lost or damaged information.

Production storage
        |
        v
Backup process
        |
        v
Timestamped recovery copy
        |
        v
Protected backup location

Backup design should define:

• What data is included
• How often copies are created
• Where copies are stored
• How long copies are retained
• How copies are protected
• How restoration is performed
• How restoration is tested

Replication vs Backup

Backup Isolation and Immutability

A recovery copy should be protected from the same identities, errors, or failures that can damage production data.

Protection can include:

• Separate security boundaries
• Restricted deletion permissions
• Retention locks where required
• Immutable backup copies
• Independent credentials
• Geographically separate storage
• Audited recovery operations

Recovery Point Objective

Recovery Point Objective, commonly abbreviated as RPO, describes the acceptable amount of data loss measured in time.

Last recoverable point:

10:00


Failure:

10:15


Potential lost interval:

15 minutes

A smaller RPO requires more frequent or more continuous protection.

Recovery Time Objective

Recovery Time Objective, commonly abbreviated as RTO, describes the target time for restoring an acceptable level of service after a disruption.

Failure detected
      |
      v
Recovery begins
      |
      v
Data restored or failover completed
      |
      v
Service becomes usable

Durability protects data, while RTO addresses how quickly the service must become operational again.

RPO vs RTO

Failure Domains

Redundant copies should be placed across failure domains appropriate to the required durability target.

Device
  |
  v
Server
  |
  v
Rack
  |
  v
Availability zone
  |
  v
Region
  |
  v
Independent backup location

Two copies on one physical device do not protect against that device's failure. Several copies in one building do not protect against the complete loss of that building.

Regional and Geographic Redundancy

Geographic redundancy stores or replicates data in locations separated from the primary storage location.

Design questions include:

• Is replication synchronous or asynchronous?
• What recent-data exposure remains?
• Can applications read from the secondary location?
• How is failover initiated?
• How is failback performed?
• Are encryption keys available in the recovery location?
• Are regulatory location requirements satisfied?

Database Durability

Database durability normally combines transaction logs, checkpoints, persistent data pages, recovery processing, and the underlying storage platform.

Application transaction
        |
        v
Database transaction log
        |
        v
Durable storage acknowledgment
        |
        v
COMMIT returned
        |
        v
Checkpoint and page persistence
        |
        v
Recovery after failure

Database durability settings can trade write latency for a weaker or stronger acknowledgment guarantee. Review platform-specific configuration before changing the default.

File-storage Durability

File-storage durability can combine protected underlying devices, file-system journaling, redundant servers, snapshots, checksums, and backups.

Application writes file
      |
      v
File-system operation
      |
      v
File metadata and data persisted
      |
      v
Storage redundancy applied
      |
      v
Backup or snapshot retained

A successful application write does not automatically prove that all buffers have reached durable media. File APIs, flush operations, mount settings, and storage guarantees must be understood.

Object-storage Durability

An object-storage system can protect data by storing redundant complete copies or encoded fragments across several devices or locations.

Object upload
      |
      v
Validate request and data
      |
      v
Store redundant representation
      |
      v
Persist object metadata
      |
      v
Acknowledge successful upload
      |
      v
Periodically verify integrity

Object versioning, retention, lifecycle policies, and independent backup can protect against additional risks such as accidental overwrite or deletion.

Database and Object-storage Workflow

An object upload and relational database update normally cannot share one ordinary local transaction.

Upload object succeeds
        |
        v
Database metadata insert fails
        |
        v
Durable object exists
without active business record

A resilient workflow can use:

• Pending metadata status
• Generated object identifiers
• Upload verification
• Idempotent finalization
• Abandoned-object cleanup
• Metadata-to-storage reconciliation

Durable Upload Workflow

1. Create pending asset metadata.

2. Generate a stable object key.

3. Upload the object.

4. Verify object existence,
   expected size,
   and checksum.

5. Mark the asset active.

6. Retain appropriate versions or backups.

7. Reconcile incomplete workflows.

Accidental-deletion Recovery

Object deleted accidentally
        |
        v
Check object versions
        |
        +--> Previous version exists:
        |       restore selected version
        |
        +--> No retained version:
                locate backup
                restore object
                verify checksum
                verify metadata reference

A deletion process should account for versions, snapshots, replicas, backups, caches, search indexes, legal holds, and business metadata.

Restore Testing

A backup is useful only when the required data can be restored correctly.

A restore test should verify:

• The backup can be located
• Decryption keys are available
• The backup can be read
• The required recovery point exists
• The restored data passes integrity checks
• Application references remain valid
• The restored service can start
• Access control remains correct
• Recovery completes within the required objective

Durability Metrics

A durability percentage communicates a statistical design target for data loss over a period. It is not a promise that an individual object can never be lost.

A simplified annual loss expectation can be expressed as:

\[ ExpectedLostObjects = StoredObjects \times AnnualLossProbability \]

Use provider-published definitions when interpreting reported durability. Confirm the measured unit, period, exclusions, service scope, and failure assumptions.

Durability and Data Retention

Durable storage preserves data, while retention rules determine how long the data should remain.

Keeping data forever is not automatically safer. Unnecessary retention can increase privacy, compliance, operational, and cost risks.

Encryption-key Durability

Encrypted durable content becomes unusable if the required decryption keys are permanently lost.

Durable encrypted backup
        |
        v
Decryption key unavailable
        |
        v
Data cannot be restored

Key-management planning should address:

• Key backup or protected redundancy
• Access control
• Rotation
• Recovery-location availability
• Auditability
• Revocation and destruction

Threats to Durable Data

Durability planning should consider:

• Hardware failure
• Software defects
• Operator mistakes
• Expired credentials
• Lost encryption keys
• Ransomware
• Unauthorized deletion
• Silent corruption
• Incorrect lifecycle policies
• Replication lag
• Incomplete backups
• Untested restoration

Durability Observability

Useful durability metrics include:

• Write acknowledgment failures
• Replication lag
• Under-replicated object or block count
• Checksum verification failures
• Repair backlog
• Storage-device failures
• Snapshot success and failure count
• Backup success and failure count
• Backup age
• Restore-test success rate
• Restore duration
• Unrecoverable-object count
• Expired-version count
• Recovery-point age

Alerting

Alert conditions can include:

• Required replica count is not satisfied
• Replication lag exceeds the approved threshold
• Backup did not complete
• No recent valid recovery point exists
• Checksum mismatches increase
• Repair backlog grows
• Restore test fails
• Backup or archive storage approaches capacity
• Retention or immutability policy changes unexpectedly

Durability-review Workflow

1. Identify the data and its business importance.
2. Define the acknowledged-write semantics.
3. Define the failures the system must tolerate.
4. Identify storage and replica failure domains.
5. Define required replication or erasure coding.
6. Define checksum and integrity verification.
7. Define snapshot and versioning policies.
8. Define backup frequency and retention.
9. Define RPO and RTO.
10. Protect recovery copies and encryption keys.
11. Implement repair and reconciliation.
12. Test failure and restoration procedures.
13. Monitor replication, backups, integrity, and recovery readiness.

Common Durability Mistakes

Recommended Test Cases

Durability Best Practices

Practice Exercise

Design a durability and recovery plan for your online learning platform.

Requirements

1. Classify course videos, documents, certificates, database records, and audit logs.
2. Define the acknowledged-write point for each data type.
3. Identify device, server, zone, and regional failure requirements.
4. Define replication or erasure-coding requirements.
5. Store and verify checksums for uploaded content.
6. Define object-versioning rules.
7. Define snapshot and backup schedules.
8. Define recovery retention periods.
9. Set RPO and RTO for critical platform data.
10. Protect backup credentials and encryption keys.
11. Detect orphaned objects and metadata records.
12. Test accidental object deletion.
13. Test database restoration.
14. Test recovery of one learner certificate.
15. Record recovery results and corrective actions.

Durability-design Template

Frequently Asked Questions