Partitioning and consistent hashing
When the data or the load outgrows one node, give every key a home it can compute from its own name. A hash ring does that, and when a node joins or leaves only the keys next to it move.
What is the difference between partitioning and sharding?
They name the same move: split one data set across several nodes so each holds a part. Sharding is the word most databases and most interviewers use for it, and partitioning is the older, more general word. Some systems use partition for a split inside one machine and shard for a split across machines, so ask which one the interviewer means if it matters. In a design round, say which key you split on and how a request finds its node; the word matters much less.
Why not use hash(key) % n to pick a node?
It spreads keys evenly, and it breaks the moment n changes. Going from n to n + 1 nodes moves about n of every n + 1 keys, because almost every remainder changes. On a thousand keys going from four nodes to five, this page counts 803 moves. A hash ring moves only the keys the new node takes over, about one in n + 1, and all of them go to the new node.
What are virtual nodes, and how many do I need?
A virtual node is one of several positions a physical node takes on the ring, each made by hashing the node name with a counter. With one position per node the gaps between positions are uneven, so one node can own several times what another owns. With many positions each node owns many small gaps and the shares even out. Real systems use from tens to a few hundred per node; the count is a trade between balance and the size of the ring table every client keeps.
When should I partition by range instead of by hash?
When the queries read ranges: all events in the last hour, all orders for one customer sorted by date. Range partitioning keeps neighbouring keys on the same node, so a range read touches one or two nodes instead of all of them. The cost is hot spots. When the key is a time or an increasing ID, every new write lands in the last range. Salting the key with a small bucket number spreads the writes and makes each range read ask every bucket.
Does partitioning fix a hot key?
No. Partitioning spreads different keys across nodes, and one key always has one home, so every read of a celebrity profile or a global counter still goes to one node. The fixes are different blocks: cache or replicate the hot value so many nodes can serve it, or split one logical key into several physical ones (a salted counter) and combine them on read.