Introduction

Storage is a foundational system-design decision. Applications store databases, operating-system volumes, uploaded documents, videos, backups, logs, reports, machine-learning datasets, and many other forms of data.

These workloads do not all require the same access model. A database may require low-level random reads and writes. A shared application may expect directories and normal file operations. A media platform may need to store billions of independently addressable objects.

The three major storage models are:

• Block storage: Exposes raw addressable blocks to a host or operating system.
• File storage: Exposes files and directories through a hierarchical file system.
• Object storage: Stores complete objects identified by keys and accessed through an API.

In your System Design curriculum, Block, File, and Object Storage is Topic 6.1 and begins the Storage, Files, Objects & Search Basics module. It is followed by durability, checksums, metadata, uploads and multipart transfer, inverted indexes, and full-text search.

Prerequisites

Understanding the Storage Stack

An application can interact with storage through several abstraction layers.

Application
    |
    +-- Database API
    |
    +-- File-system API
    |
    +-- Object-storage API
    |
    v
Operating system or storage client
    |
    v
Local or network storage service
    |
    v
Physical storage devices

The selected abstraction determines which operations the application can perform and which responsibilities belong to the application, operating system, or storage service.

Block Storage

Block storage divides a storage volume into fixed-size addressable blocks. The storage layer reads or writes blocks without inherently understanding files, directories, database rows, images, or document formats.

Block device

+---------+---------+---------+---------+
| Block 0 | Block 1 | Block 2 | Block 3 |
+---------+---------+---------+---------+
| Block 4 | Block 5 | Block 6 | Block 7 |
+---------+---------+---------+---------+

An operating system can place a file system on the block device, or a database can manage its own data structures over the available volume.

Block-storage Characteristics

• Data is addressed as blocks
• The block layer does not inherently provide filenames or directories
• A host can format the volume with a file system
• Applications can perform random reads and writes
• Only selected blocks need to be changed during an in-place update
• The attached host commonly controls the file system placed on the volume

Common Block-storage Use Cases

• Database data and transaction-log volumes
• Virtual-machine boot and data disks
• Container persistent volumes
• File-system volumes
• Low-latency transactional workloads
• Applications requiring frequent random updates

File Storage

File storage organizes data into files and directories. Applications access content through paths and file-level operations.

/
+-- courses/
|   +-- system-design/
|       +-- chapter-01.pdf
|       +-- chapter-02.pdf
|
+-- learners/
|   +-- learner-1042/
|       +-- certificate.pdf
|
+-- media/
    +-- lesson-01.mp4

The file system manages filenames, directory structures, metadata, and file-level access operations.

File-storage Characteristics

• Hierarchical files and directories
• Path-based access
• File metadata such as size and timestamps
• File and directory permissions
• Shared access through supported network file protocols
• Partial reads and writes through file offsets
• Familiar file-system semantics for applications

Common File-storage Use Cases

• Shared application directories
• User home directories
• Document-management systems
• Content-production workflows
• Legacy applications that require file paths
• Shared configuration or application assets
• Workloads requiring file locking or partial file updates

Object Storage

Object storage stores content as independently addressable objects. Each object generally contains data, associated metadata, and a key or identifier.

Bucket or container
    |
    +-- Key:
    |   courses/system-design/lesson-01.mp4
    |
    +-- Object data:
    |   video bytes
    |
    +-- Metadata:
        content type
        content length
        checksum
        creation time
        custom attributes

Object storage is normally accessed through an API rather than through ordinary local file-system operations.

Object-storage Characteristics

• Objects are addressed through keys
• Objects can contain rich system and custom metadata
• Storage is commonly organized into buckets or containers
• Namespaces commonly appear hierarchical while remaining key-based
• Objects are accessed through APIs
• Complete-object replacement is common for updates
• The model is suitable for large quantities of unstructured data

Common Object-storage Use Cases

• User-uploaded images and videos
• Course documents and learning resources
• Backups and archives
• Application logs
• Data-lake datasets
• Static website assets
• Generated reports and exports
• Machine-learning training data

Block vs File vs Object Storage

Similar Content, Different Interfaces

The same logical content can be stored using different models, but the application interface and operational behaviour change.

Video content as block storage:

Stored inside a file system
on an attached volume.


Video content as file storage:

Stored at a shared path such as
/media/course-42/lesson-7.mp4


Video content as object storage:

Stored under an object key such as
courses/42/lessons/7/video.mp4

The correct model depends on how the content must be written, shared, replaced, distributed, protected, retained, and retrieved.

Update Semantics

Block Update

Existing volume data:

[Block A] [Block B] [Block C]


Modified region:

Update Block B


Result:

[Block A] [New Block B] [Block C]

File Update

Open file

Move to byte offset

Write changed bytes

Close file

Object Update

Read or prepare replacement object

Upload new object content

Replace object associated with key

Namespace Design

File Namespace

/tenant-17/courses/course-42/lesson-7/video.mp4

Each path component is part of a directory hierarchy.

Object Namespace

tenant-17/courses/course-42/lesson-7/video.mp4

The separators can make the key appear hierarchical, but the storage service can treat the complete value as an object key.

Good Object-key Qualities

• Stable over the object's lifecycle
• Unique within the required scope
• Independent of mutable display names
• Safe to log under the applicable policy
• Compatible with expected prefix-based organization
• Not dependent on untrusted user input alone

Object Metadata

Object metadata describes the stored content.

{
  "objectKey": "courses/42/lessons/7/video.mp4",
  "contentType": "video/mp4",
  "contentLength": 52428800,
  "checksum": "stored-checksum-value",
  "courseId": "42",
  "lessonId": "7",
  "visibility": "private"
}

Metadata can support:

• Content-type handling
• Download naming
• Integrity verification
• Lifecycle rules
• Retention classification
• Search indexing
• Access-control decisions
• Operational diagnostics

Metadata limits, mutability, search behaviour, and supported fields depend on the object-storage service.

Store File Metadata in a Database

Applications commonly store large file bytes in object storage while keeping business metadata in a relational database.

CREATE TABLE course_assets
(
    asset_id BIGINT PRIMARY KEY,
    tenant_id BIGINT NOT NULL,
    course_id BIGINT NOT NULL,
    lesson_id BIGINT NULL,
    object_key VARCHAR(500) NOT NULL,
    original_file_name VARCHAR(255) NOT NULL,
    content_type VARCHAR(100) NOT NULL,
    content_length BIGINT NOT NULL,
    checksum_value VARCHAR(200) NOT NULL,
    asset_status VARCHAR(30) NOT NULL,
    created_at TIMESTAMP NOT NULL,

    CONSTRAINT uq_course_assets_object_key
        UNIQUE
        (
            tenant_id,
            object_key
        ),

    CONSTRAINT ck_course_asset_length
        CHECK (content_length >= 0)
);

The database row represents the application's business record. The object key identifies the stored binary content.

Direct and Shared Access

This table provides common architectural directions, not universal rules. Validate the actual storage product's behaviour and workload requirements.

Object Upload Flow

Client selects file
      |
      v
Application validates upload request
      |
      v
Application creates upload authorization
      |
      v
Client uploads object
      |
      v
Storage verifies or stores content
      |
      v
Application verifies completion
      |
      v
Metadata record becomes active

The architecture can route upload bytes through the application or authorize a client to upload directly to object storage.

Object Download Flow

Client requests asset
      |
      v
Application authenticates caller
      |
      v
Application authorizes object access
      |
      v
Application streams content
or creates temporary access
      |
      v
Client downloads object

Possession of an object key should not automatically grant access to private content.

HTTP Object Upload Example

PUT /course-assets/courses/42/lesson-7/video.mp4 HTTP/1.1
Host: storage.example.com
Content-Type: video/mp4
Content-Length: 52428800
X-Content-Checksum: stored-checksum-value

[BINARY CONTENT]

This is a conceptual example. Authentication, API paths, checksum fields, conditional requests, size limits, and response semantics depend on the selected object-storage service.

PHP Upload-validation Example

<?php

declare(strict_types=1);

function validateUploadedAsset(
    array $file,
    int $maximumBytes,
    array $allowedMimeTypes
): void {
    $error =
        $file['error'] ??
        UPLOAD_ERR_NO_FILE;

    if ($error !== UPLOAD_ERR_OK) {
        throw new RuntimeException(
            'The upload did not complete successfully.'
        );
    }

    $size =
        (int)($file['size'] ?? 0);

    if ($size <= 0 ||
        $size > $maximumBytes) {

        throw new RuntimeException(
            'The uploaded file size is invalid.'
        );
    }

    $temporaryPath =
        (string)($file['tmp_name'] ?? '');

    $mimeDetector =
        new finfo(
            FILEINFO_MIME_TYPE
        );

    $detectedMimeType =
        $mimeDetector->file(
            $temporaryPath
        );

    if (!is_string($detectedMimeType) ||
        !in_array(
            $detectedMimeType,
            $allowedMimeTypes,
            true
        )) {

        throw new RuntimeException(
            'The uploaded file type is not allowed.'
        );
    }
}

File extensions and client-provided content types are insufficient by themselves. Production systems also require authorization, secure filenames, malware controls where appropriate, quotas, rate limits, integrity verification, and protected storage access.

Secure Upload Design

Upload controls can include:

• Authentication
• Object-level authorization
• Maximum file size
• Allowed content types
• Filename normalization
• Generated storage keys
• Integrity checks
• Malware scanning where required
• Tenant quotas
• Rate limiting
• Retention and deletion rules
• Audit records

Object key is copied directly
from an untrusted uploaded filename.

Tenant ID
+
Generated asset ID
+
Approved file extension or content category

Access Control

Storage authorization should reflect the data's business ownership and classification.

Request:

Download learner certificate


Authorization checks:

- Caller is authenticated
- Caller belongs to trusted tenant scope
- Asset record exists
- Caller owns the certificate
  or has an approved administrative permission
- Asset status permits download

Storage buckets, shared file systems, and block volumes should follow least privilege. Public access should be enabled only when explicitly required.

Temporary Object Access

Private object access can be granted through a short-lived signed request or by streaming content through an authorized application endpoint.

Application verifies permission
        |
        v
Application creates temporary access
        |
        v
Client receives bounded access
        |
        v
Temporary authorization expires

The duration, allowed operation, object key, content headers, and other conditions should be as restricted as practical.

Encryption

Storage security can protect data:

• In transit between clients and storage
• At rest on storage infrastructure
• Through managed or customer-controlled encryption keys
• Through application-level encryption where required

Encryption does not replace authorization, key management, auditability, backups, retention, or secure deletion.

Replication Is Not Backup

Several copies can improve availability and tolerance of infrastructure failures, but repeated copies can also reproduce accidental deletion or corruption.

Object Versioning

Object key:

courses/42/lesson-7/notes.pdf


Version 1:
Original notes


Version 2:
Corrected notes


Version 3:
Latest published notes

Versioning can help recover previous object content, but it increases stored data and requires retention, lifecycle, and deletion rules.

Lifecycle Management

Storage lifecycle policies can move or remove data according to age, state, type, or business retention rules.

New object
    |
    v
Frequently accessed tier
    |
    v
Infrequently accessed tier
    |
    v
Archive tier
    |
    v
Delete after approved retention period

Lifecycle rules should reflect legal, audit, recovery, and business requirements. Lower-cost tiers can have different access time and retrieval charges.

Storage Cost Dimensions

Storage cost can include:

• Provisioned capacity
• Consumed storage
• Read and write operations
• Data retrieval
• Network transfer
• Snapshots and retained versions
• Replication
• Performance units
• Backup and archive retention

Compare the complete access and lifecycle pattern rather than storage capacity alone.

Performance Dimensions

Large-file Uploads

Very large objects can require multipart or resumable transfer.

Large file
    |
    v
Split into numbered parts
    |
    v
Upload parts independently
    |
    v
Retry failed parts
    |
    v
Request completion
    |
    v
Storage assembles final object

Multipart transfer is covered in detail later in this module.

Integrity Verification

Applications can calculate and store a checksum to verify that retrieved or transferred data matches the expected content.

sha256sum course-video.mp4

Checksum algorithms, server-side validation, multipart checksums, and metadata fields depend on the selected storage service. Checksums are covered in detail later in this module.

Consistency Expectations

Storage systems can define different visibility and update semantics for reads, writes, overwrites, listings, metadata, and replicas.

The application should verify:

• When a newly written object becomes readable
• When an overwrite becomes visible
• Whether listing reflects the latest changes
• How concurrent writes are resolved
• Whether conditional writes are supported
• How cached content is invalidated

Do not infer one storage service's consistency semantics from another service's behaviour.

Conditional Object Updates

Where supported, a conditional write can prevent one client from overwriting content that changed after it was read.

Client reads object version V1

Another client writes version V2

First client attempts update:

Write only if current version is still V1

Result:

Conditional write fails
instead of overwriting V2 silently

This is the object-storage equivalent of optimistic concurrency for a supported object operation.

Orphaned Objects

Storage and database updates normally do not participate in one ordinary local transaction.

Upload object succeeds
        |
        v
Database insert fails
        |
        v
Object exists without application record

The reverse can also occur:

Database row commits
        |
        v
Object upload fails
        |
        v
Application record points to missing content

Possible workflow controls include:

• Pending metadata state
• Upload completion verification
• Idempotent finalization
• Cleanup of abandoned uploads
• Reconciliation jobs
• Deletion queues

Safe Upload-finalization Workflow

1. Create pending asset record.

2. Generate storage key.

3. Upload object.

4. Verify object exists,
   size matches,
   and integrity checks pass.

5. Mark asset active.

6. Clean up incomplete uploads
   after a documented expiry period.

The exact flow depends on whether uploads pass through the application or go directly from the client to storage.

Deletion Workflow

Deletion requested
      |
      v
Authorize deletion
      |
      v
Mark business record pending deletion
      |
      v
Delete or queue storage deletion
      |
      v
Verify outcome
      |
      v
Finalize metadata state

Deletion design should account for retention holds, object versions, replicas, backups, caches, audit requirements, and retryable failures.

Course-platform Storage Example

Content-delivery Architecture

Learner
   |
   v
Application authorization
   |
   v
Content-delivery layer
   |
   v
Object storage

Frequently downloaded static content can be distributed through a content delivery layer while private access remains governed by the application's authorization contract.

Caching rules must account for updates, deletion, privacy, tenant scope, and time-limited authorization.

Search Is a Separate Concern

Storage systems preserve content, but they do not automatically provide the application-level full-text search experience required by users.

Stored document
      |
      v
Extract text and metadata
      |
      v
Build search index
      |
      v
Execute user query
      |
      v
Return authorized search results

Inverted indexes and full-text search are covered later in this module.

Storage Observability

Useful storage metrics include:

• Stored capacity
• Object or file count
• Request rate
• Read and write latency
• Throughput
• Error and timeout rate
• Upload failure rate
• Integrity-verification failures
• Abandoned multipart uploads
• Orphaned metadata or objects
• Lifecycle transition count
• Deletion backlog
• Data-transfer volume
• Storage and retrieval cost

Storage Troubleshooting Workflow

1. Identify the storage model and service.
2. Identify the affected file, volume, bucket, or object key safely.
3. Confirm authentication and authorization.
4. Confirm network connectivity and endpoint configuration.
5. Check capacity, quota, and object-size limits.
6. Check read, write, and list error responses.
7. Check storage latency and throughput.
8. Verify metadata and content type.
9. Verify checksum or integrity evidence.
10. Check object version and lifecycle state.
11. Check application metadata for stale references.
12. Check caches and content-delivery behaviour.
13. Compare runtime behaviour with the documented storage contract.

Common Storage-design Mistakes

Recommended Test Cases

Storage Best Practices

Practice Exercise

Design the storage architecture for your online learning platform.

Requirements

1. Store the relational course and quiz database.
2. Store lesson videos, PDFs, source-code files, and presentations.
3. Store learner profile images.
4. Generate and store course certificates.
5. Support private course content.
6. Support large resumable video uploads.
7. Store file metadata in the relational database.
8. Generate stable object keys.
9. Validate content type and file size.
10. Verify uploaded checksums.
11. Define object versioning and lifecycle rules.
12. Define backup and restore procedures.
13. Detect orphaned database rows and objects.
14. Define short-lived download authorization.
15. Measure storage, retrieval, and network costs.

Storage-selection Template

Frequently Asked Questions