Database Sharding & Partitioning

### The Limits of Vertical Database Scaling

Eventually, a single master database server will reach its physical limits. When read-replicas aren't enough to handle write-heavy workloads, we must scale the database horizontally through Sharding.

### What is Sharding?

Sharding involves splitting a single logical database across multiple physical database nodes. Each node (shard) holds a distinct subset of the data.

### Sharding Strategies

- **Algorithmic (Hash) Sharding:** Uses a hash function on a partition key (e.g., User ID) to determine the shard. Evenly distributes data but makes adding new shards difficult. - **Directory/Range Sharding:** Uses a lookup table to map ranges of data (e.g., Users A-M on Shard 1, N-Z on Shard 2). Easier to rebalance, but can lead to hot-spots if ranges aren't evenly accessed.

Sharding introduces immense operational complexity, including challenges with distributed joins and transactions, and should generally be used as a last resort.