Prerequisites

Why Designate a Leader

Many coordination problems become dramatically simpler when a single node is responsible for a decision. Rather than negotiating every action, participants defer to one coordinator.

The Two Requirements

Election Mechanisms

Lock-Based Election

The most common practical approach delegates the hard part to a consensus-backed coordination service. The application competes for a lock and treats acquisition as leadership.

class LeaderElection {
    constructor(coordinator, config) {
        this.coordinator = coordinator;
        this.lockKey = config.lockKey;
        this.nodeId = config.nodeId;
        this.leaseSeconds = config.leaseSeconds;
        this.isLeader = false;
        this.fencingToken = null;
    }

    async campaign() {
        const result = await this.coordinator.acquire({
            key: this.lockKey,
            holder: this.nodeId,
            ttlSeconds: this.leaseSeconds
        });

        if (!result.acquired) {
            this.isLeader = false;
            return { leader: false, currentHolder: result.holder };
        }

        this.isLeader = true;
        this.fencingToken = result.fencingToken;
        this.startRenewal();

        return { leader: true, token: this.fencingToken };
    }

    startRenewal() {
        const intervalMs = (this.leaseSeconds * 1000) / 3;

        this.renewalTimer = setInterval(async () => {
            const renewed = await this.coordinator.renew({
                key: this.lockKey,
                holder: this.nodeId,
                ttlSeconds: this.leaseSeconds
            });

            if (!renewed.ok) {
                this.onLeadershipLost(renewed.reason);
            }
        }, intervalMs);
    }

    onLeadershipLost(reason) {
        clearInterval(this.renewalTimer);
        this.isLeader = false;
        this.fencingToken = null;
        this.stopLeaderWork(reason);
    }
}

-- Conditional acquisition, atomic by construction
UPDATE cluster_leadership
SET holder_id     = :node_id,
    fencing_token = fencing_token + 1,
    acquired_at   = CURRENT_TIMESTAMP,
    expires_at    = CURRENT_TIMESTAMP + INTERVAL '15 seconds'
WHERE resource   = :resource
  AND (expires_at < CURRENT_TIMESTAMP OR holder_id = :node_id)
RETURNING fencing_token, expires_at;

-- Zero rows returned means another node holds valid leadership

The Stale Leader Problem

This is the defining hazard of leader election. A leader that loses contact with the cluster continues believing it leads, because nothing has told it otherwise.

Fencing Tokens

A fencing token is a monotonically increasing number issued on each leadership grant. The protected resource records the highest token it has seen and rejects anything lower.

-- Resource-side fencing: stale writers are rejected
UPDATE shard_assignment
SET assignment    = :new_assignment,
    fencing_token = :incoming_token,
    updated_at    = CURRENT_TIMESTAMP
WHERE shard_id     = :shard_id
  AND fencing_token <= :incoming_token;

-- Zero rows affected means a deposed leader attempted a write

class FencedResource {
    constructor() {
        this.highestToken = 0;
        this.state = null;
    }

    write(value, token) {
        if (token < this.highestToken) {
            return {
                accepted: false,
                reason: "stale fencing token",
                presented: token,
                required: this.highestToken
            };
        }

        this.highestToken = token;
        this.state = value;

        return { accepted: true, token };
    }
}

Leases and Clock Assumptions

A lease grants leadership for a bounded period. Its safety rests on an assumption that deserves scrutiny: that clock drift stays within a known bound.

class MonotonicLease {
    constructor(leaseSeconds, safetyMarginMs) {
        this.leaseMs = leaseSeconds * 1000;
        this.marginMs = safetyMarginMs;
        this.acquiredAtMonotonic = null;
    }

    acquire() {
        this.acquiredAtMonotonic = performance.now();
    }

    isSafelyHeld() {
        if (this.acquiredAtMonotonic === null) return false;

        const elapsed = performance.now() - this.acquiredAtMonotonic;

        return elapsed < (this.leaseMs - this.marginMs);
    }

    assertLeadership() {
        if (!this.isSafelyHeld()) {
            throw new LeadershipExpiredError(
                "lease margin exhausted, refusing to act"
            );
        }
    }
}

Failover Timing

The lease duration is a direct trade-off between how quickly failure is detected and how often healthy leaders are wrongly replaced.

Becoming and Ceasing to Be Leader

Transitions are where most bugs live. Both directions require deliberate handling.

class LeaderWorker {
    async runCycle() {
        if (!this.election.isLeader) return;

        this.election.lease.assertLeadership();

        const work = await this.fetchPendingWork();

        for (const item of work) {
            this.election.lease.assertLeadership();

            const result = await this.processItem(item, {
                fencingToken: this.election.fencingToken
            });

            if (!result.accepted && result.reason === "stale fencing token") {
                this.election.onLeadershipLost("fenced by resource");
                return;
            }
        }
    }

    async shutdown() {
        this.stopped = true;
        await this.drainInFlight();

        if (this.election.isLeader) {
            await this.election.release();
        }
    }
}

Partition Behavior

Scaling Beyond One Leader

A single leader is a throughput ceiling. Where the workload partitions naturally, electing a leader per partition preserves the coordination guarantee while restoring parallelism.

Monitoring

Verification

describe("leader election safety", function () {
    it("rejects writes from a deposed leader", async function () {
        const first = await electLeader("node-a");
        const firstToken = first.fencingToken;

        await faultInjector.partition("node-a");
        const second = await electLeader("node-b");

        expect(second.fencingToken).toBeGreaterThan(firstToken);

        const staleWrite = await resource.write("value-x", firstToken);

        expect(staleWrite.accepted).toBe(false);
        expect(staleWrite.reason).toBe("stale fencing token");
    });

    it("never permits two simultaneous leaders", async function () {
        const observations = await runChaosScenario({
            durationMs: 120_000,
            partitions: true,
            pauses: true,
            restarts: true
        });

        for (const moment of observations) {
            expect(moment.activeLeaders.length).toBeLessThanOrEqual(1);
        }
    });

    it("self-demotes when renewal fails", async function () {
        const leader = await electLeader("node-a");

        await faultInjector.blockCoordinator("node-a");
        await sleep(leaseMs);

        expect(leader.isLeader).toBe(false);
    });
});

Common Design Mistakes

System Design Interview Discussion

Design Checklist

Knowledge Check

Summary

Leader election designates one node to coordinate decisions that must not be made concurrently. It has two requirements: at most one leader at a time, and eventually some leader. Safety dominates, because an outage is recoverable while split-brain corruption often is not.

Election itself is usually delegated to a consensus-backed coordination service. The genuinely hard problem is the stale leader: a node partitioned or paused continues believing it leads, and no amount of self-checking can make it certain at the moment its write reaches a resource.

Fencing tokens are the robust answer. A monotonic token issued per leadership grant, validated at the resource, rejects writes from any deposed leader without requiring that leader to detect anything. Leases add time bounds but depend on clock assumptions that pauses and backward jumps can violate.

Lease duration trades detection speed against election churn, and leadership should be scoped to the narrowest unit requiring exclusivity so a single coordinator does not become a throughput ceiling for work that partitions cleanly.