Introduction

A horizontally scaled application runs several instances behind a load balancer. One request can reach Instance A, while the next request from the same client can reach Instance B.

Request 1
    -> Instance A


Request 2
    -> Instance C


Request 3
    -> Instance B

This routing remains correct only when the selected instance has access to all the information required to process the request.

A stateless application instance does not depend on private data left in its memory or local disk by an earlier client request. Every request contains sufficient context, or sufficient lookup information, for any healthy instance to process it.

Persistent and shared state still exists. It can be stored in:

• Relational databases
• Document databases
• Shared session stores
• Distributed caches
• Object storage
• Message queues
• Durable workflow stores
• Signed client tokens

Statelessness makes it easier to add, remove, restart, replace, and rebalance application instances without losing authoritative user data.

Prerequisites

What Is State?

State is information that describes the current or previous condition of a user, request, workflow, resource, or system.

Examples include:

• A user's authenticated session
• Items in a shopping cart
• A learner's course progress
• An enrollment workflow status
• A multipart upload's completed parts
• A background job's processing checkpoint
• An idempotency record
• A database transaction

State is not inherently a problem. The design question is where that state lives, who owns it, how long it must survive, and which application instances need access to it.

Stateful Application Instance

A stateful application instance stores request-critical information in its private memory or local disk and expects later requests to return to that same instance.

Login request
      |
      v
Instance A
      |
      v
Session stored only
in Instance A memory


Later request
      |
      v
Instance B
      |
      v
Session not found

The application's correctness now depends on Instance A continuing to exist and receiving the user's future requests.

Problems

• Load-balancer routing becomes constrained
• Instance failure can lose user state
• Scale-in can interrupt active users
• Deployments require session preservation
• Uneven user activity can create uneven server load
• Automatic replacement becomes more difficult

Stateless Application Instance

A stateless application instance processes the request without depending on private state created by a previous request.

Request
   |
   +-- Credential or session identifier
   +-- Required request parameters
   +-- Idempotency key when applicable
   |
   v
Load Balancer
   |
   +-- Instance A
   +-- Instance B
   +-- Instance C
          |
          v
Shared data services

Any healthy instance can validate the request, retrieve the required shared state, execute the operation, and return the response.

Stateful vs Stateless

Stateless Does Not Mean Memoryless

Every running application uses memory. A stateless instance can maintain:

• Connection pools
• Compiled code
• Configuration
• Metrics
• Read-through caches
• Reusable HTTP clients
• Temporary request buffers

These values are compatible with statelessness when losing them does not remove authoritative user or business data.

Instance-local cache miss
      |
      v
Read authoritative shared source
      |
      v
Rebuild cache entry
      |
      v
Return correct response

Shopping cart stored only
in one process dictionary


Instance terminates
      |
      v
Shopping cart is lost

Why Statelessness Enables Horizontal Scaling

Initial deployment:

Load Balancer
      |
      +-- Instance A
      +-- Instance B


Traffic increases:

Load Balancer
      |
      +-- Instance A
      +-- Instance B
      +-- Instance C
      +-- Instance D


Traffic decreases:

Load Balancer
      |
      +-- Instance A
      +-- Instance B

New instances do not need to receive existing user-session memory from older instances. Removed instances do not take authoritative session or business data with them.

Benefits of Statelessness

• Flexible request routing
• Simpler horizontal scaling
• Safer instance replacement
• Improved tolerance of individual compute failures
• Simpler rolling deployments
• Safer autoscaling
• Reduced dependence on sticky sessions
• More consistent backend utilization
• Faster recovery from unknown local state

Statelessness Trade-offs

• External state access adds network calls
• The session store can become a shared dependency
• Token revocation can require additional design
• Shared-state consistency must be defined
• External storage requires capacity and recovery planning
• Large client-carried tokens increase request size
• Distributed workflows require idempotency and durable progress records

Statelessness moves state management out of disposable compute. It does not eliminate the need to design state correctly.

Session State

A session provides continuity across several requests from a client.

Login request
      |
      v
Authentication succeeds
      |
      v
Session established
      |
      v
Later requests identify
the same authenticated context

The main session design question is where the session state is stored.

Pattern 1: Local In-memory Session

Instance A memory:

session-123
    userId = 1042
    tenantId = 17
    expiresAt = session-expiry

This is simple for one server but creates instance affinity.

Request reaches Instance B
      |
      v
Instance B has no session-123
      |
      v
Authentication continuity fails

Pattern 2: Sticky Sessions

Sticky sessions configure the load balancer to keep routing related requests to the same backend.

User A
    -> Instance 1
    -> Instance 1
    -> Instance 1


User B
    -> Instance 2
    -> Instance 2
    -> Instance 2

Possible Benefit

• Can support an existing application that stores sessions locally
• Can reduce the immediate need for session-store migration

Limitations

• Backend failure can strand local session state
• Heavy users can produce uneven load
• Scale-in requires affinity-aware draining
• Deployment and rebalancing become harder
• Affinity does not make local state durable

Pattern 3: Shared Session Store

The client carries an opaque session identifier. Every application instance uses that identifier to retrieve session data from a shared store.

Client request:

Cookie:
sessionId=session-123


Load Balancer
      |
      +-- Instance A
      +-- Instance B
      +-- Instance C
              |
              v
      Shared Session Store
              |
              v
      Read session-123

Benefits

• Any application instance can handle the request
• Session data survives one application-instance failure
• Sessions can be invalidated centrally
• Session expiration can be enforced in the shared store

Trade-offs

• Every session lookup can add a network operation
• The session store requires scaling and high availability
• Store failure can affect all application instances
• Session-key distribution and expiration require monitoring

Pattern 4: Signed Client Token

A signed token carries approved claims that any application instance can validate.

GET /api/courses/42 HTTP/1.1
Host: api.example.com
Authorization: Bearer signed-access-token

The server verifies token integrity and validates the required issuer, audience, lifetime, and authorization information.

Benefits

• No application-instance session memory
• Any authorized instance can validate the token
• Can reduce central session lookups

Trade-offs

• Immediate revocation can require additional mechanisms
• Large tokens increase headers and network traffic
• Stale claims can remain valid until token expiry or revocation
• Private information must not be placed carelessly in client-visible tokens
• Signing keys and validation configuration require secure rotation

Session-pattern Comparison

Local-file State

Files created on one instance's local filesystem are not automatically available on another instance.

Upload request
      |
      v
Instance A
      |
      v
Save file to:

/var/app/uploads/course.pdf


Download request
      |
      v
Instance B
      |
      v
File not found

Local ephemeral storage can be used for temporary processing when the application does not depend on that instance retaining the file.

Files that must survive or be shared should be moved to appropriate durable storage.

Shared File and Object Storage

Upload request
      |
      v
Any application instance
      |
      v
Validate metadata
      |
      v
Store object in shared storage
      |
      v
Persist object metadata
      |
      v
Any instance can retrieve it

The database or metadata service can store:

• Tenant ID
• Asset ID
• Object key
• Content type
• Size
• Checksum
• Lifecycle state
• Authorization metadata

Direct-upload Pattern

1. Client requests upload authorization.

2. Application validates the caller.

3. Application creates a controlled upload target.

4. Client uploads directly to object storage.

5. Application verifies and finalizes the asset.

The application remains the authority for upload permission, object naming, size, content type, lifecycle, and finalization even when bytes do not pass through one application instance.

Shopping-cart or Enrollment State

Instance A memory:

learner-1042
    selectedCourse = 42
    paymentOption = card


Instance A terminates
      |
      v
Selection is lost

Enrollment draft:

tenantId = 17
learnerId = 1042
courseId = 42
status = pending


Stored in:

Authoritative database
or approved shared workflow store

Business state requiring durability, auditability, or concurrency control belongs in an appropriate authoritative datastore.

Background Jobs

A web request should not depend on a particular application process continuing to run after sending the response.

Web request returns success
      |
      v
Instance starts local background thread
      |
      v
Instance is scaled in
      |
      v
Work disappears

Web request
      |
      v
Validate operation
      |
      v
Persist authoritative state
      |
      v
Publish or enqueue durable work
      |
      v
Any eligible worker processes it

Durable queueing allows another worker to process the job after an instance fails or is removed.

Worker Statelessness

A worker should not depend exclusively on an in-memory processing position.

Worker receives message
      |
      v
Read authoritative processing state
      |
      v
Perform idempotent operation
      |
      v
Persist result
      |
      v
Acknowledge message

If the worker fails before acknowledgment, the message can be delivered again. The operation therefore needs safe redelivery behaviour.

Idempotency

Idempotency ensures that repeating an operation with the same identity does not create a second unintended business effect.

POST /api/enrollments HTTP/1.1
Host: api.example.com
Authorization: Bearer access-token
Idempotency-Key: unique-operation-key
Content-Type: application/json

{
  "courseId": 42
}

The idempotency record must be stored outside one replaceable application instance when retries can reach different instances.

Conceptual Idempotency Table

CREATE TABLE idempotency_records
(
    tenant_id BIGINT NOT NULL,
    operation_key VARCHAR(150) NOT NULL,
    request_hash VARCHAR(128) NOT NULL,
    operation_status VARCHAR(30) NOT NULL,
    response_code INT NULL,
    response_body TEXT NULL,
    expires_at TIMESTAMP NOT NULL,

    PRIMARY KEY
    (
        tenant_id,
        operation_key
    )
);

Exact schema, retention, locking, and response-storage choices depend on the operation and database.

Database Connections

A stateless application can still maintain a local database connection pool. The pool is replaceable infrastructure state, not authoritative business state.

Instance A
    -> Connection Pool A


Instance B
    -> Connection Pool B


Instance C
    -> Connection Pool C


All pools connect to
the approved database endpoint.

Horizontal scaling multiplies connection pools.

A simplified connection estimate is:

\[ MaximumConnections = InstanceCount \times PoolSizePerInstance \]

Reserve database capacity for background workers, administrative operations, deployment tasks, monitoring, and failure headroom.

Local Cache in a Stateless Service

A process-local cache is compatible with statelessness when it is disposable and can be rebuilt from an authoritative source.

Request arrives
      |
      v
Check local cache
      |
      +-- Hit:
      |      return cached value
      |
      +-- Miss:
             read shared source
             store temporary local copy
             return value

Do not assume local caches contain the same data on every instance.

If immediate invalidation is required, use a suitable shared cache, versioning, invalidation event, short expiration policy, or authoritative lookup according to the correctness requirement.

Soft State vs Hard State

Classifying state helps determine whether it can remain local or must move to durable shared storage.

WebSockets and Long-lived Connections

A service can be logically stateless while maintaining long-lived network connections.

Client
    |
    | WebSocket connection
    v
Instance A

The live connection is attached to Instance A for its lifetime. The system must define:

• Connection routing
• Instance drain behaviour
• Client reconnection
• Subscription recovery
• Message ordering
• Presence or connection-directory state
• Missed-message handling

Durable subscriptions and business messages should not exist only in the connection-handling process.

Notifications Example

Client connected to Instance A
      |
      v
Notification event published
      |
      v
Shared messaging or connection routing layer
      |
      v
Instance A sends notification


Instance A fails
      |
      v
Client reconnects to Instance C
      |
      v
Subscription is restored
from external state

Graceful Shutdown

Statelessness simplifies instance removal, but in-progress requests and connections still require draining.

Instance selected for removal
      |
      v
Mark readiness as false
      |
      v
Stop receiving new requests
      |
      v
Complete permitted in-flight work
      |
      v
Stop background consumers
      |
      v
Close connections
      |
      v
Terminate instance

Statelessness makes recovery from termination easier. It does not mean that active requests can be interrupted carelessly.

Containers and Statelessness

Containers and application instances can be created, restarted, rescheduled, and replaced.

Container Instance A
      |
      v
Terminates


Orchestrator
      |
      v
Starts Instance D


Instance D
      |
      v
Loads configuration
      |
      v
Connects to shared services
      |
      v
Passes readiness
      |
      v
Receives traffic

Any data written only into a container's ephemeral filesystem can disappear when the container is replaced.

Immutable and Replaceable Instances

Replaceable instances are normally created from approved code, images, and configuration rather than being manually repaired in place.

Configuration or application change
      |
      v
Build new approved artifact
      |
      v
Create replacement instances
      |
      v
Wait for readiness
      |
      v
Shift traffic
      |
      v
Drain and retire old instances

This model works best when important user and business state is external to the compute instance.

Tenant Isolation

Externalized state must preserve tenant boundaries.

Session key:

tenant-17:session:session-123


Cart key:

tenant-17:learner-1042:cart


Upload metadata:

tenantId = 17
assetId = asset-981

Tenant identifiers must come from trusted authentication and authorization context. A client-provided tenant ID alone must not determine access.

Security Considerations

When externalizing state:

• Protect session identifiers from theft
• Use secure cookie attributes where applicable
• Encrypt sensitive data according to policy
• Set bounded session and token lifetimes
• Protect token-signing and validation keys
• Prevent cross-tenant session access
• Do not place unnecessary sensitive claims in client tokens
• Audit session creation and revocation where required
• Protect shared stores from direct public access

PHP Shared-session Example

<?php

declare(strict_types=1);

final class SessionContext
{
    public function __construct(
        private SessionStore $sessionStore
    ) {
    }

    public function getAuthenticatedSession(
        string $sessionId
    ): array {
        if ($sessionId === '') {
            throw new RuntimeException(
                'The session identifier is missing.'
            );
        }

        $session =
            $this->sessionStore->findById(
                $sessionId
            );

        if ($session === null) {
            throw new RuntimeException(
                'The session is invalid or expired.'
            );
        }

        if ($session['expiresAt'] <= time()) {
            throw new RuntimeException(
                'The session has expired.'
            );
        }

        return $session;
    }
}

SessionStore is a conceptual abstraction. Production code needs protected session identifiers, expiration, revocation, tenant validation, secure cookies, error handling, and approved storage-client configuration.

Stateless Request-handler Example

<?php

declare(strict_types=1);

final class CourseController
{
    public function __construct(
        private AuthenticationContext $authentication,
        private CourseRepository $courses
    ) {
    }

    public function getCourse(
        int $courseId
    ): array {
        $caller =
            $this->authentication
                ->getCurrentCaller();

        $course =
            $this->courses->findAuthorizedCourse(
                tenantId:
                    $caller['tenantId'],

                courseId:
                    $courseId,

                callerId:
                    $caller['callerId']
            );

        if ($course === null) {
            throw new RuntimeException(
                'The course was not found.'
            );
        }

        return $course;
    }
}

The controller does not depend on another request having run on the same process. It reconstructs the required caller and course context from request information and shared services.

Conceptual Kubernetes Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: course-api

spec:
  replicas: 3

  template:
    spec:
      containers:
        - name: course-api
          image: approved-course-api-image

          ports:
            - containerPort: 8080

          readinessProbe:
            httpGet:
              path: /health/ready
              port: 8080

          livenessProbe:
            httpGet:
              path: /health/live
              port: 8080

          env:
            - name: SESSION_STORE_ENDPOINT
              valueFrom:
                secretKeyRef:
                  name: application-connections
                  key: session-store-endpoint

This is conceptual configuration. Production deployments require approved secret handling, network policies, timeouts, resources, probe thresholds, graceful termination, and platform-specific settings.

Statelessness Assessment

To determine whether an application instance is stateless, ask:

1. Can the next request safely reach another instance?
2. Can this instance terminate without losing authoritative user data?
3. Where are sessions stored?
4. Where are uploaded files stored?
5. Where is workflow progress stored?
6. Where are background jobs recorded?
7. Where are idempotency records stored?
8. Can local cache contents be discarded safely?
9. Can long-lived clients reconnect to another instance?
10. Can an instance be rebuilt from approved configuration?

Finding Hidden Local State

Review application use of:

• Static variables
• Global arrays
• In-memory dictionaries
• Local session files
• Local upload directories
• Local job queues
• Temporary workflow files
• Process-local locks
• In-memory idempotency keys
• Instance-specific scheduled tasks

Some of these are safe temporary implementation details. Others can reveal correctness dependencies on one specific instance.

Migration toward Statelessness

1. Inventory all process-local and local-disk state.
2. Classify each item as authoritative, soft, temporary, or derived.
3. Identify the required lifetime and consistency.
4. Select the appropriate shared or durable store.
5. Move session state first if it prevents load-balanced routing.
6. Move shared files to durable storage.
7. Move background work to durable queues.
8. Persist workflow and idempotency records.
9. Make local caches disposable.
10. Remove unnecessary session affinity.
11. Test instance termination during active traffic.
12. Test scale-out, scale-in, and rolling deployment.

Failure Test

1. Authenticate a controlled test user.

2. Begin a normal user workflow.

3. Identify the serving instance.

4. Remove that instance safely.

5. Send the next request through the load balancer.

6. Verify that another instance reconstructs the context.

7. Verify that no authoritative state is lost.

The test should also cover in-flight operations, idempotent retries, long-lived connections, uploads, and background jobs.

Observability

Useful statelessness metrics include:

• Requests per instance
• Session-store request rate
• Session-store latency
• Session lookup failures
• Session expiration and revocation count
• Sticky-session distribution
• Local-cache hit rate
• Shared-cache latency
• Instance replacement count
• Scale-out and scale-in events
• Connection-pool usage per instance
• Queue redelivery count
• Idempotency replay count
• Upload-resume failures
• WebSocket reconnection count
• Graceful-drain duration

Alert Conditions

Alert when:

• Session-store availability falls below its requirement
• Session lookup latency increases
• Users lose sessions after instance replacement
• One backend receives disproportionate sticky-session traffic
• Local filesystem usage grows unexpectedly
• Background jobs disappear or repeatedly redeliver
• Idempotency records fail to persist
• Database connections grow unexpectedly after scale-out
• Scale-in interrupts uploads or long-lived connections
• New application instances cannot become ready

Troubleshooting Workflow

1. Identify which user or workflow state was lost.
2. Identify the instance that handled the preceding request.
3. Identify the instance that handled the failing request.
4. Check whether session affinity was expected.
5. Locate the authoritative state.
6. Check shared-session or token validation.
7. Check local filesystem dependencies.
8. Check cache-key and tenant-scope behaviour.
9. Check queue acknowledgment and redelivery.
10. Check idempotency persistence.
11. Check scale-in and graceful-drain events.
12. Check deployment changes and configuration.
13. Reproduce the operation while forcing requests to different instances.

Common Statelessness Mistakes

Recommended Test Cases

Statelessness Best Practices

Practice Exercise

Make your online learning platform's course and enrollment APIs stateless.

Requirements

1. Run at least two API instances behind a load balancer.
2. Route successive requests to different instances.
3. Move authenticated-session data out of process memory.
4. Choose between shared sessions and signed client tokens.
5. Define session expiry and revocation.
6. Move course uploads to shared object storage.
7. Persist upload metadata in an authoritative store.
8. Move content-processing jobs to a durable queue.
9. Make job processing idempotent.
10. Store idempotency records outside the application instance.
11. Make local caches disposable.
12. Set a database connection budget per instance.
13. Add readiness and graceful shutdown.
14. Terminate one instance during an authenticated user workflow.
15. Verify that another instance continues the workflow.
16. Test tenant isolation in sessions, caches, and object keys.
17. Monitor session, queue, connection, and scale-in behaviour.

State-placement Template

Frequently Asked Questions