System Design Lab
File sync is two systems wearing one coat: a chatty metadata service and a fat block store that scale on different axes.
Change active users, files per user, file size, edit rate, devices per user, chunk size, and regions. The design grows from one box to a notification service that pushes changes to every device, content-addressed chunking with dedup, a sharded metadata database, and multi-region block replication for sharing and collaboration.
Guided walkthrough
Reason about it one step at a time
Takeaway
Normal evolution scenarios
Click left to right for the intended demo path. Each card changes the workload inputs.
Recommended shape
Current architecture path
Devices
Client device watches the local folder, uploads changed chunks, and applies remote changes
Sync API
Sync API handles chunk uploads and downloads and commits file changes
Notification tells every other device of a user when a file changes so it can pull
Metadata
Metadata svc stores the file tree, versions, and permissions; the chatty source of truth
Chunk index maps content hashes to stored blocks so identical chunks are stored once
Block store
Block store durable content-addressed store of every unique file chunk
Replicas copies blocks across regions so uploads and downloads stay local
Async
Indexer fans change events to devices and builds search and sharing indexes off the hot path
Bottlenecks
Metadata QPS
Notification fan-out
Block storage
Bytes saved by dedup
Conflict / version pressure
Why this changes
Decision tradeoffs
Source-backed rules
These are the durable system-design claims behind the model. The exact slider thresholds are deliberately labeled as teaching assumptions.
Dropbox separates a metadata service from block storage
Dropbox runs Magic Pocket as a dedicated exabyte-scale block store while file metadata lives in a separate, differently scaled service.
Dropbox EngineeringFiles are split into content-addressed blocks
Content-addressable storage names each block by a hash of its contents, so identical blocks are stored once β the basis for block-level dedup.
Content-addressable storageContent-defined chunking enables cross-file deduplication
The Low-Bandwidth File System showed that hashing variable-size content chunks lets a sync system transfer and store only the chunks that actually changed.
LBFS (SOSP 2001)Long-polling lets clients learn of changes promptly without busy polling
A held-open request (long-poll or server-sent stream) lets the server notify a device of a change as it happens instead of the device polling on a timer.
MDN Web DocsTeaching assumptions
- Metadata QPS is approximated from edits and the devices that must be notified per change, not measured traffic.
- Single-node metadata read and storage budgets are conservative teaching numbers, not vendor limits.
- Dedup savings use a flat fraction of total bytes; real savings depend heavily on how much content is actually shared.