Design a Distributed File Synchronization & Cloud Storage Engine (Dropbox / Google Drive)
A cloud file synchronization engine breaks files into 4MB cryptographic chunks, using Content-Addressable Storage to eliminate duplicate data and sync delta changes across devices.
Live Architecture Studio
Edit components, modify labels, add databases, or redraw connections directly on this canvas:
Loading Dropbox Cloud Sync Blueprint...
Mounting vector diagram elements, nodes, and capacity metrics
Loading Dropbox Cloud Sync Blueprint...
Mounting vector diagram elements, nodes, and capacity metrics
1. Problem & Challenge
When a user edits a 200MB video file or complex spreadsheet, uploading the entire file wastes massive network bandwidth. The system must split files into independent chunks, deduplicate blocks, and resolve concurrent sync conflicts gracefully.
2. Core Building Blocks & Responsibilities
👉 Desliza la tabla para ver roles y responsabilidades| Component | Role | Plain-English Explanation |
|---|---|---|
| Desktop Sync Daemon | OS File Watcher | Monitors file system change events (inotify/FSEvents), chunking files into 4MB blocks and computing SHA-256 hashes. |
| Metadata Metaserver | Namespace & Recipes | Maintains directory hierarchies, file versions (rev_id), and maps each file to an ordered list of chunk hashes. |
| Blockserver & CAS Storage | Magic Pocket Blob Store | Pure Content-Addressable Storage (CAS) keyed by SHA-256. Blocks are immutable and erasure-coded with Reed-Solomon parity. |
| Notification Push Service | Device Waker | Maintains persistent HTTP/2 SSE connections to peer devices, waking them immediately when a file change is committed. |
| Pipelined Transfer Engine | Streaming Chunks | Streams chunks concurrently: Device B begins downloading Chunk 1 the instant Device A finishes uploading it. |
3. Step-by-Step Request Flow
Local Change Detected
Client daemon watches directory, chunks modified file, and calculates SHA-256 hash for each 4MB block.
Check Hashes with Metaserver
Client sends list of chunk hashes. Metaserver replies with only the missing hashes not yet present in storage.
Upload Missing Chunks
Client uploads only new unique blocks to the Blockserver CAS store.
Commit Manifest & Notify Peers
Metaserver increments file rev_id and Notification Service wakes peer devices to download new chunks.
4. Architectural Trade-offs
Fixed 4MB Chunking vs Content-Defined Chunking (Rabin)
Chosen: Content-Defined Chunking (CDC)
Rationale: In fixed 4MB chunking, inserting 1 byte at the start of a 100MB file shifts all boundaries, forcing all 25 chunks to re-upload. Content-defined chunking sets boundaries dynamically, saving 90%+ bandwidth.
Distributed File Locking vs Optimistic Revision OCC
Chosen: Optimistic Concurrency Control (OCC)
Rationale: Distributed file locks are fragile across intermittent laptop Wi-Fi connections. Using revision numbers (rev_id) lets the server reject conflicting commits and create friendly "(Conflicted Copy)" local duplicate files.
Interview Tip
Mention CAS Garbage Collection: When a file is deleted, its chunks cannot be removed immediately because other users or versions might reference identical SHA-256 blocks. Explain two-phase asynchronous mark-and-sweep garbage collection with a 30-day grace period.
Explore Related System Blueprints
TinyURL Shortener
A URL shortener converts a long link (like a 100-character article URL) into a compact 7-character key (like tinyurl.com/xyz123) and redirects visitors in under 15 milliseconds.
API Rate Limiter
A rate limiter acts as a digital bouncer at the door of your API, ensuring each client stays within their allowed request limits (e.g. 100 requests per minute) and blocking abusive traffic.
Video Streaming CDN
Streaming high-definition video to millions of smart TVs and mobile phones requires breaking large 10GB video files into tiny 5-second chunks, encoding each into 20 different resolutions, and caching them right inside local ISP networks.
Uber Dispatch Engine
A real-time geospatial dispatch system matches riders with the most optimal nearby drivers using 64-bit H3 hexagonal indexing and 2-second batch optimization, minimizing city-wide pickup ETA and driver idle time.
Stripe Payments Ledger
A resilient financial payments architecture guarantees strict consistency (CP system) using cryptographic idempotency reservation, double-entry balanced postings, and sharded balance locks.
Figma Multiplayer Engine
A real-time multiplayer document engine uses stateful sticky session routing and server-authoritative operational ordering to sync 2D scene graphs across worldwide collaborators without locking.