System Design: Distributed Systems Fundamentals

Master core distributed systems concepts including physical vs logical time, consensus algorithms (Raft, Paxos), commit protocols (2PC/3PC), distributed locks, and fault tolerance.

1. What Makes a System "Distributed"?

A distributed system is a collection of autonomous computing nodes connected via a network that coordinate their actions by passing messages, presenting themselves to end-users as a single coherent system.

Core Advantages of Distributed Systems

  • Horizontal Scalability:Add commodity compute nodes to handle linear growth in traffic and data storage.
  • High Availability & Fault Tolerance:If a single server hardware host crashes, remaining cluster nodes take over transparently.
  • Low Latency (Geo-Distribution):Deploy server nodes physically closer to global users across multiple data centers.

2. Distributed System Challenges (Network Partitions & Clock Sync)

Unlike single-machine software execution, distributed systems operate over unreliable physical networks subject to packet loss, message reordering, and physical clock drift.

Distributed Systems Challenges: Network Partitions & Logical Clocks

System Design

Visualizing physical clock skew challenges and logical event ordering using Lamport and Vector clocks

100%
Loading system design canvas…

Clock Synchronization & Clock Drift

Physical quartz clocks on different server machines drift due to temperature variations. Protocols like NTP sync clocks within milliseconds, but physical timestamps cannot establish strict event causality.

Logical Clocks (Lamport Timestamps & Vector Clocks)

Logical clocks order events using monotonically increasing counter tuples passed along message payloads, determining 'happened-before' relationships without relying on physical wall clocks.

3. Leader Election & Consensus Algorithms (Paxos, Raft)

Distributed consensus is the fundamental problem of getting a group of independent nodes to agree on a single data value or state transition sequence.

Raft Consensus Protocol: Leader Election & Quorum Log Replication

System Design

Tracing candidate election terms, AppendEntries RPC heartbeats, and majority quorum log commits

100%
Loading system design canvas…

Raft Consensus Algorithm

Decomposes consensus into Leader Election, Log Replication, and Safety. Raft uses randomized election timeouts to ensure single candidates win elections rapidly.

Paxos Consensus Algorithm

The classic consensus protocol using Two-Phase rounds (Prepare/Promise and Accept/Accepted) to reach agreement despite node crashes and message delays.

Examine the Raft leader election state machine simulation below to see randomized timeouts and vote requests in action.

Raft Consensus Leader Election Simulation

Implementing candidate term increments, randomized election timers, and majority quorum vote checks

4. Distributed Locking & Commit Protocols (2PC vs 3PC)

Atomic transaction processing across multiple distinct database nodes requires structured commit protocols and distributed mutex locks.

Two-Phase Commit (2PC) vs Three-Phase Commit (3PC) Protocol Flow

System Design

Comparing blocking coordinator 2PC prepare/commit phases against non-blocking 3PC PreCommit states

100%
Loading system design canvas…

Two-Phase Commit (2PC)

Phase 1 (Prepare): Coordinator asks all nodes if they are ready to commit. Phase 2 (Commit): If all vote YES, coordinator sends COMMIT. If the coordinator crashes during Phase 2, nodes block holding locks.

Three-Phase Commit (3PC)

Adds a non-blocking PreCommit phase between Prepare and Commit. Allows participants to time out and safely abort if the coordinator disappears.

Distributed Locks (Redlock & Fencing Tokens)

Acquires mutex locks across distributed Redis nodes using atomic leases. Includes monotonically increasing fencing tokens to reject stale writes from delayed processes.

Review the Node.js Distributed Lock engine below demonstrating Redis atomic key leases and atomic Lua release scripts.

Distributed Lock Engine (Redlock Pattern & Fencing Tokens)

Acquiring atomic leases with Redis SET NX PX and releasing ownership using atomic Lua scripts

5. Gossip Protocol, Split-Brain & Byzantine Fault Tolerance

Decentralized cluster management requires peer-to-peer state propagation algorithms and partition protection mechanisms.

Gossip Protocol Dissemination & Split-Brain Quorum Partitioning

System Design

Visualizing O(log N) peer gossip state dissemination and majority quorum split-brain protection

100%
Loading system design canvas…

Gossip Protocol (Epidemic Dissemination)

Nodes periodically pick random peers to exchange cluster membership state. Spreads state changes across thousands of nodes in logarithmic O(log N) time with zero single point of failure.

Split-Brain Problem & Quorum Fencing

Occurs when a network partition cuts a cluster in half, causing nodes on both sides to elect separate leaders. Prevented by requiring a Majority Quorum (N/2 + 1) for write authorization.

Byzantine Fault Tolerance (BFT)

Addresses scenarios where nodes can not only fail or crash, but also send corrupt or malicious data (e.g. blockchain networks). Reaches consensus if over 2/3 of nodes are honest (3f + 1 total nodes required to tolerate f faulty nodes).

Distributed Systems Fundamentals Knowledge Verification

1. What defines a distributed system?

2. Why are physical wall-clock timestamps unreliable in distributed systems?

3. How does the Raft consensus algorithm select a new leader node?

4. What is the main drawback of the Two-Phase Commit (2PC) protocol?

5. What is the Split-Brain problem in distributed clusters?