Sharding

Hash or range partitioning: which should I pick?

Hash when every operation addresses one key and no query needs keys in order, because hashing spreads keys evenly and a range scan has to ask every partition. Range when queries read keys in order, such as a time window or the next page of a sorted list, and accept that sequential keys pile onto the newest range until it splits. When you need both, hash the partition key and keep a sorted key inside each partition.

Why is hash modulo N a bad way to place data?

Because the remainder changes for almost every key when N changes. Going from 30 nodes to 31, hash modulo moves 96.8 percent of the data, where a ring moves 3.23 percent, 30 times less. On a store holding 6 TB that is 5.81 TB copied against 0.19 TB.

How do I fix a hot key in a sharded store?

Climb the ladder from the cheapest rung and stop at the first that fits. A cache or read replicas for a hot read; salting the key into sub-keys, or aggregating locally, for a hot write; a dedicated partition for one tenant that outgrows the others; and coalescing with back-pressure as the last guard. Detect it first with the request rate per partition, because the cluster average hides it.

What does salting a hot key cost?

Every read of the key. The writes to one counter are spread over several sub-keys, so each partition takes an even share, but a read now asks every sub-key and adds the parts. With four sub-keys a read of the counter becomes four reads, which is worth it when the key is written far more often than it is read.

What is a fixed partition count, and what does it cap?

You create many more partitions than nodes on day one and assign whole partitions to nodes, so a key never re-hashes and a new node takes whole partitions from the others. With 1,000 partitions and 30 nodes, a new node takes 32 of them. The count caps the cluster: a node owns at least one partition, so 1,000 partitions allow at most 1,000 nodes, and changing the count later means re-hashing everything.