System design rounds are a key part of interviews for experienced software engineers. These system design interview questions cover scalability basics, load balancing and caching for those starting out, then databases and sharding, consistency, message queues, rate limiting and classic designs such as a URL shortener and chat system for senior engineers.
30 questions · Updated
Fundamentals, definitions and simple scenarios. Good for freshers and warm-ups.
Vertical scaling means adding more power, such as CPU and memory, to one server; it is simple but has a hardware limit and a single point of failure. Horizontal scaling means adding more servers and spreading the load across them; it scales much further and improves availability, but needs stateless services, load balancing and handling of distributed data.
A load balancer distributes incoming requests across multiple servers, so no single server is overloaded, and stops sending traffic to unhealthy servers using health checks.
Algorithms include round robin, least connections and IP hash.
Layer 4 balancers route by IP and port; layer 7 balancers route by HTTP details such as path or headers.
Caching stores frequently used data in faster storage to reduce latency and load on databases.
You can cache in the browser, at a CDN for static content, in the application's memory, in a distributed cache such as Redis or Memcached, and in the database's own buffers.
The hard parts are invalidation and keeping data fresh, handled with TTLs and update strategies.
Clarify functional and non-functional requirements (scale, latency, availability, consistency), estimate traffic and storage, define the main APIs and data model, draw a high-level design with clients, load balancers, services, databases and caches, then go deep into the hardest parts, such as scaling the database or handling failures. Discuss trade-offs openly and keep checking with the interviewer.
Latency is how long one request takes, such as 120 milliseconds to load a page, usually measured at percentiles such as p95 and p99 rather than averages.
Throughput is how many requests the system handles per unit of time, such as 5,000 requests per second.
Improving one can hurt the other, for example batching increases throughput but adds latency.
A content delivery network caches content on servers close to users around the world, so static files, images, videos and even some API responses load faster and the origin server handles less traffic.
CDNs also absorb traffic spikes and DDoS attacks.
Content is refreshed using cache headers, versioned file names or explicit invalidation.
DNS translates domain names into IP addresses.
It can also route traffic: weighted routing for gradual migrations, latency-based or geolocation routing to send users to the nearest region, and failover routing with health checks for disaster recovery.
DNS changes propagate slowly because of caching, controlled by the TTL.
A stateless service keeps no client data between requests, so any server can handle any request, which makes horizontal scaling and replacing servers easy; session data lives in a shared store such as Redis or in tokens. A stateful service keeps data in memory or on local disk, such as databases or WebSocket servers, which requires careful routing, replication and failover.
In leader-follower replication, all writes go to the leader, which copies changes to followers that serve reads, increasing read capacity and providing failover.
Replication can be synchronous (safer, slower) or asynchronous (faster, but followers may lag, so users might not immediately see their own writes).
Multi-leader and leaderless replication allow writes in several places at the cost of conflict handling.
An SLI (service level indicator) is a measured metric, such as the percentage of successful requests or p99 latency.
An SLO (service level objective) is the target for it, such as 99.9% of requests succeeding each month.
An SLA (service level agreement) is a contract with customers with consequences if targets are missed.
99.9% availability allows about 43 minutes of downtime per month.
Clear resource-based endpoints with the right HTTP methods and status codes, consistent naming and error format, pagination for lists, versioning, authentication and rate limiting, and idempotency for operations that may be retried, such as payments, using an idempotency key. For internal service-to-service calls, gRPC gives efficient, strongly typed communication.
Applied problems, trade-offs and questions about your own projects.
In a distributed system, when a network partition happens, you must choose between consistency (every read gets the latest write) and availability (every request gets a response).
Partition tolerance is unavoidable in practice.
Banking systems usually prefer consistency, while social feeds prefer availability with eventual consistency.
PACELC adds that, even without partitions, you trade latency against consistency.
SQL databases suit structured data with relationships, complex queries and strong consistency through transactions, such as orders and payments.
NoSQL databases suit flexible schemas, very high write throughput and horizontal scale, such as key-value stores for sessions, document stores for catalogues and wide-column stores for time-series.
Many systems use both for different data.
Message queues such as Kafka, RabbitMQ or SQS let services communicate asynchronously: producers publish messages and consumers process them at their own pace.
They decouple services, absorb traffic spikes, allow retries and enable event-driven architectures.
Consumers must handle duplicate messages idempotently because most queues guarantee at-least-once delivery.
Sharding splits data across multiple database servers by a shard key, such as user ID, so each server holds a part of the data and handles part of the load.
Strategies include range, hash and directory-based sharding.
Challenges are choosing a key that spreads load evenly, cross-shard queries and joins, rebalancing when adding shards, and hot spots.
Read replicas are a simpler first step for read-heavy loads.
With simple modulo hashing, adding or removing a server changes where almost every key goes.
Consistent hashing places servers and keys on a ring, and each key belongs to the next server clockwise, so adding or removing a server only moves the keys next to it.
Virtual nodes spread load evenly.
It is used in distributed caches, databases such as Cassandra and DynamoDB, and load balancers.
Cache-aside: the app reads the cache and loads from the database on a miss.
Write-through: writes go to the cache and database together, keeping them consistent.
Write-back: writes go to the cache and are saved to the database later, which is fast but can lose data.
Eviction policies such as LRU (least recently used), LFU (least frequently used) and TTL expiry decide what to remove when the cache is full.
A monolith is one deployable application, simpler to build, test and deploy, and the right start for most products and small teams.
Microservices split the system into independently deployable services owned by separate teams, allowing independent scaling and releases, but add network calls, distributed data, complex debugging and operational overhead.
A modular monolith is a good middle ground until scale demands splitting.
An API gateway is the single entry point for clients in front of many services.
It handles routing, authentication, rate limiting, request transformation, caching, logging and sometimes aggregating responses from several services.
It keeps cross-cutting concerns out of individual services.
Examples are Kong, AWS API Gateway, Apigee and Spring Cloud Gateway.
Start from users and behaviour: for example 10 million daily active users each making 20 requests gives 200 million requests a day, roughly 2,300 per second on average, and perhaps 5 to 10 times that at peak.
Multiply by object size to estimate storage and bandwidth, such as 1 KB per record times 100 million records per day giving about 100 GB per day.
Round numbers aggressively; the goal is the order of magnitude.
Network calls fail, so clients retry, which can duplicate actions such as charging a card twice.
Make operations idempotent: the client sends a unique idempotency key and the server stores the result for that key and returns it again on a retry.
Retries should use exponential backoff with jitter and a limit, and circuit breakers stop calling a service that is clearly down.
When a service must update its database and publish an event, doing both separately can leave them inconsistent if one fails. With the outbox pattern, the service writes the business change and the event into an outbox table in the same database transaction, and a separate process or change data capture tool such as Debezium reads the outbox and publishes the events reliably.
System design, deep internals, leadership and tough follow-ups.
Requirements: create short links, redirect quickly, handle far more reads than writes.
I generate a unique ID with a counter or distributed ID generator and encode it in base62 to get a 7-character code, store code-to-URL mappings in a key-value store or sharded database, and serve redirects with 301 or 302 through a cache such as Redis and a CDN since reads dominate.
I add rate limiting, link expiry and asynchronous click analytics through a queue.
Clients keep persistent WebSocket connections to chat servers, with a service tracking which server each user is connected to.
A message is stored in a database partitioned by conversation (such as Cassandra), then pushed to the recipient's server, or queued and sent through push notifications if they are offline.
Delivery and read receipts are acknowledgements.
Group messages fan out to members, media goes to object storage with a CDN, and end-to-end encryption happens on clients.
I would use a token bucket or sliding window algorithm, keyed by user, API key or IP, with counters in Redis updated atomically using INCR with expiry or a Lua script, so all API servers share limits.
It runs in an API gateway or middleware.
Over-limit requests get HTTP 429 with a Retry-After header.
I would decide whether to fail open or closed if Redis is unavailable, and allow different limits per plan.
Users post content stored in a posts database and media in object storage behind a CDN.
Feeds can be built on write (fan-out to followers' precomputed feeds in Redis when someone posts), which makes reading fast, or on read (gather posts from followed accounts when the feed is opened), which suits celebrities with millions of followers.
Most systems use a hybrid, then rank posts with a model, paginate with cursors and cache heavily.
Services publish notification requests to a queue.
A notification service checks user preferences, rate limits and quiet hours, renders templates and sends each notification to channel workers for push (APNs and FCM), SMS, email and in-app delivery, through separate queues so one slow channel does not block others.
It tracks delivery status, retries failures with backoff, deduplicates messages and records analytics.
Uploads go directly to object storage with pre-signed URLs.
A processing pipeline using queues and workers transcodes each video into several resolutions and bitrates and splits it into small segments for adaptive streaming (HLS or DASH), so players switch quality based on bandwidth.
Segments are served through CDNs.
Metadata, search, recommendations, view counts and comments are separate services with their own storage.
Distributed transactions with two-phase commit are slow and fragile across services, so most systems use the saga pattern: a sequence of local transactions, each publishing an event that triggers the next step, with compensating actions to undo earlier steps if a later one fails, such as refunding a payment when stock cannot be reserved. Sagas can be choreographed with events or orchestrated by a central coordinator.
Driver apps send location updates every few seconds to a location service that stores current positions in an in-memory geospatial index, such as Redis with geohashes or H3 cells.
A matching service finds nearby available drivers for a ride request and offers the trip, using locking so one driver is not matched twice.
Trips, payments, pricing (including surge based on demand in each area), ETAs and notifications are separate services, with WebSockets for live tracking.
See all Software Engineer interview questions, covering Data Structures, Algorithms, OOP, System Design, Operating Systems and more.