Prerequisites

Why This Is Not Like a Mutex

A mutex within a process works because the operating system knows definitively which thread holds it, and a thread cannot vanish while holding it without the kernel noticing.

Efficiency Versus Correctness

This distinction determines everything that follows. Before choosing an implementation, decide which of these two problems you are actually solving.

Why Locks Alone Cannot Guarantee Exclusion

The lock service may be perfectly correct and still fail to provide exclusion, because the gap between holding a lock and acting on it is not instantaneous.

Fencing Is the Actual Solution

Since the client cannot know whether it still holds the lock, the resource must decide. A fencing token makes the resource the arbiter of exclusion.

-- Lock service issues a monotonic token per acquisition
UPDATE distributed_locks
SET holder_id     = :client_id,
    fencing_token = fencing_token + 1,
    expires_at    = CURRENT_TIMESTAMP + (:ttl_ms * INTERVAL '1 millisecond')
WHERE lock_name = :lock_name
  AND (expires_at < CURRENT_TIMESTAMP OR holder_id = :client_id)
RETURNING fencing_token;

-- The resource rejects anything stale
UPDATE inventory
SET quantity      = quantity - :amount,
    fencing_token = :token
WHERE sku           = :sku
  AND fencing_token <= :token
  AND quantity      >= :amount;

async function performGuardedWork(lockService, resource, work) {
    const acquisition = await lockService.acquire({
        lockName: work.lockName,
        clientId: work.clientId,
        ttlMs: work.ttlMs
    });

    if (!acquisition.acquired) {
        return { done: false, reason: "lock held elsewhere" };
    }

    try {
        const outcome = await resource.apply(work.payload, {
            fencingToken: acquisition.fencingToken
        });

        if (!outcome.accepted) {
            return {
                done: false,
                reason: "fenced",
                presented: acquisition.fencingToken,
                required: outcome.requiredToken
            };
        }

        return { done: true, result: outcome.result };
    } finally {
        await lockService.release({
            lockName: work.lockName,
            clientId: work.clientId,
            fencingToken: acquisition.fencingToken
        });
    }
}

Implementation Approaches

The Single-Node Cache Hazard

async function acquireSimpleLock(cache, lockName, clientId, ttlMs) {
    const acquired = await cache.set(lockName, clientId, {
        notExists: true,
        expiryMs: ttlMs
    });

    return { acquired, clientId };
}

async function releaseSimpleLock(cache, lockName, clientId) {
    const script = `
        if redis.call('get', KEYS[1]) == ARGV[1] then
            return redis.call('del', KEYS[1])
        else
            return 0
        end
    `;

    return await cache.eval(script, 1, lockName, clientId);
}

The Redlock Debate

Redlock acquires a lock on a majority of independent cache nodes, aiming to survive individual node failure without full consensus. Whether it is safe has been actively disputed.

Lock Duration and Renewal

class RenewingLock {
    constructor(service, config) {
        this.service = service;
        this.config = config;
        this.held = false;
        this.renewalsRemaining = config.maxRenewals;
    }

    async acquire() {
        const result = await this.service.acquire({
            lockName: this.config.lockName,
            clientId: this.config.clientId,
            ttlMs: this.config.ttlMs
        });

        if (!result.acquired) return result;

        this.held = true;
        this.token = result.fencingToken;
        this.acquiredAt = performance.now();
        this.startRenewal();

        return result;
    }

    startRenewal() {
        this.timer = setInterval(async () => {
            if (this.renewalsRemaining <= 0) {
                return this.abandon("maximum hold time exceeded");
            }

            const renewed = await this.service.renew({
                lockName: this.config.lockName,
                clientId: this.config.clientId,
                token: this.token,
                ttlMs: this.config.ttlMs
            });

            if (!renewed.ok) {
                return this.abandon(renewed.reason);
            }

            this.renewalsRemaining -= 1;
        }, this.config.ttlMs / 3);
    }

    heldSafely() {
        const elapsed = performance.now() - this.acquiredAt;
        return this.held && elapsed < (this.config.ttlMs - this.config.marginMs);
    }

    abandon(reason) {
        clearInterval(this.timer);
        this.held = false;
        this.onLockLost(reason);
    }
}

Contention and Fairness

A lock serializes access, so throughput on the protected resource becomes independent of how many clients exist. Contention is not a tuning problem; it is the design.

Alternatives Worth Preferring

Most distributed locks exist because a simpler mechanism was overlooked. These alternatives avoid the coordination entirely.

-- Optimistic concurrency: no lock, conflict detected on write
UPDATE documents
SET content = :new_content,
    version = version + 1,
    updated_at = CURRENT_TIMESTAMP
WHERE document_id = :document_id
  AND version     = :expected_version;

-- Atomic conditional decrement: exclusion without a lock
UPDATE inventory
SET quantity = quantity - :amount
WHERE sku      = :sku
  AND quantity >= :amount;

Failure Modes

Monitoring

Verification

describe("distributed lock safety", function () {
    it("rejects work from a client that paused past expiry", async function () {
        const clientA = await acquireLock("resource-x", "client-a");
        const staleToken = clientA.fencingToken;

        await faultInjector.pauseProcess("client-a", ttlMs * 2);
        const clientB = await acquireLock("resource-x", "client-b");

        expect(clientB.fencingToken).toBeGreaterThan(staleToken);

        const staleWrite = await resource.apply(payload, {
            fencingToken: staleToken
        });

        expect(staleWrite.accepted).toBe(false);
    });

    it("does not release a lock held by another client", async function () {
        const clientA = await acquireLock("resource-x", "client-a");
        await sleep(ttlMs + 100);

        const clientB = await acquireLock("resource-x", "client-b");
        await releaseLock("resource-x", "client-a");

        const stillHeld = await lockService.holder("resource-x");
        expect(stillHeld).toBe("client-b");
    });

    it("maintains at most one holder under chaos", async function () {
        const timeline = await runChaosScenario({
            durationMs: 180_000,
            clients: 8,
            pauses: true,
            partitions: true,
            serviceFailover: true
        });

        for (const moment of timeline) {
            expect(moment.acceptedWriters.length).toBeLessThanOrEqual(1);
        }
    });
});

Common Design Mistakes

System Design Interview Discussion

Design Checklist

Knowledge Check

Summary

A distributed lock is fundamentally weaker than a local mutex. Holder liveness cannot be determined, communication fails routinely, and an arbitrary delay can occur between acquiring the lock and acting on the resource it protects.

The essential first question is whether the lock provides efficiency or correctness. If simultaneous holders merely duplicate work, a simple implementation suffices. If they corrupt data, no lock protocol is sufficient on its own, because process pauses have no upper bound.

Correctness requires fencing tokens validated at the resource, which shifts enforcement to the only party that can reliably decide. This reframes the lock as an optimization preventing wasteful contention, with the token providing the actual guarantee.

Most distributed locks exist because a simpler mechanism was overlooked. Database transactions, optimistic concurrency, partitioned ownership, idempotent operations, and atomic primitives each avoid the coordination dependency, throughput ceiling, and failure modes that a lock introduces.