Design a file-sync service
Fifty million users, ten gigabytes each, and a save that must reach every other device within seconds. The round turns on splitting bytes from metadata: chunks named by their hash in an object store, and a per-folder change log that devices read from their own cursor.
The round this design follows
Why split files into chunks instead of uploading the whole file?
Because an edit changes a small part of a file, and a whole-file upload resends all of it. At about 833 saves a second at the peak, sending a byte-weighted 50 MB file each time is 41.7 GB/s of ingest; sending the one or two changed 4 MB chunks, less the three in ten already stored, is 4.67 GB/s. Chunks also make an upload resumable one chunk at a time, and naming each chunk by its hash gives deduplication for free.
Should devices poll for changes or be told?
Told. Forty-five million connected devices polling every five seconds is nine million requests a second, and nearly all of them come back empty, because the whole service only sees about 833 saves a second at the peak. A notification service holds one long-lived connection per device and sends a short message with the folder's new sequence number; the device then reads the change log after its own cursor. The cursor is the source of truth, so a lost push costs a delay and never a missed change.
How do you keep two edits of the same file from overwriting each other?
Every commit carries the version the device started from, and the metadata store applies it only if that is still the current version, in one conditional update. The first device to commit wins. The second finds the version has moved and its edit is saved as a new file beside the first, a conflicted copy, so nothing is lost and the owner merges the two by hand.
Where do the file bytes live, and where does the metadata live?
The bytes are 350 PB of immutable chunks, so they live in an object store keyed by the chunk's hash, which is cheap per byte and needs no transactions because a chunk never changes. The metadata is about 100 TB of file rows, manifests and change rows that must change together, so it lives in a relational store sharded by folder into 25 shards, where one transaction covers a save.
What happens when a folder is shared by a thousand people?
Each save in it sends three thousand devices back to the same shard for the new change rows, which at two saves a second is six thousand reads a second against a shard that serves three thousand. The fix is a cache of the change log's tail in front of the shard: change rows never change once written and every device asks for the same few, so the cache answers them and the shard sees the misses.