System Design Walkthrough: Designing a URL Shortener

A URL shortener looks trivial: store a mapping, return a redirect. The trivial version hides real decisions: how to mint billions of unique keys without a bottleneck, which HTTP status code to return and what it does to analytics, how to serve tens of thousands of redirects per second from write-once, heavily skewed data, and what to do when someone uses the domain to host phishing links.

This walkthrough designs a URL shortener from requirements and estimates through the API, key generation, storage, the read path, redirect semantics, analytics, abuse, scaling and failure. The caching decisions lean on the caching strategies guide.

# Requirements

Functional: create a short link for a long URL (optional custom alias and expiry), redirect to the destination, report click counts, and disable or delete a link.

Non-functional: redirects must be fast and highly available, because a broken redirect breaks someone else's page, email or printed material; reads matter more than creates. Codes should not be trivially enumerable. Analytics may be delayed or approximate, but redirects may never wait on them.

# Scale estimates

These are assumptions chosen for round arithmetic, not a measured workload.

• →New links: 100 million per month.
• →Read to write ratio: 100 to 1, so 10 billion redirects per month.
• →Retention: 5 years.
• →Peak to average: 5 times.

A month has 30 × 86,400 = 2,592,000 seconds.

The 500 bytes per record assumes a roughly 100-byte URL plus code, owner, timestamps, flags and index overhead. Three terabytes and 200 writes per second are modest. The hard parts are the read rate and key generation, not raw volume.

Key length. A base62 alphabet (0-9, A-Z, a-z) gives 62^6 = 56.8 billion codes at six characters and 62^7 = 3.52 trillion at seven. Six billion links would fit in six characters but leave under ten-fold headroom; seven characters leave about 587-fold. The design uses seven.

Cache sizing. Click traffic is heavily skewed in practice, so assume the hottest 1 percent of links (60 million) take most reads: 60M × 500 bytes = 30 GB, which fits in a small Redis cluster. Treat the skew as a hypothesis and validate it with hit-ratio data.

# API

POST   /v1/links              { "url": "...", "alias": "optional", "expires_at": "optional" }
                              -> 201 { "code": "aB3xK9q", "short_url": "https://sho.rt/aB3xK9q" }
GET    /{code}                -> 302 Location: <destination>   (404 or 410 if unknown, expired or disabled)
DELETE /v1/links/{code}       -> 204
GET    /v1/links/{code}/stats -> 200 { "clicks": ..., "by_day": [...], "by_country": [...] }

Creation requires an API key and is rate-limited. Support an Idempotency-Key header so a retried POST does not mint a second code. Validate the destination: only http and https, a maximum length, and nothing pointing back at your own domain.

# Generating keys

The key generator is the main design decision. There are three realistic options.

## Option 1: counter plus base62

Take the next integer from a counter and encode it. No collisions, the shortest possible keys, a few lines of logic. The costs: the counter is a coordination point, and sequential codes are enumerable and reveal your volume. At 200 writes per second a database sequence copes easily, so the bottleneck is theoretical until you go multi-region.

## Option 2: hash the URL

Hash the long URL and encode enough of the digest to fill seven base62 characters. It is stateless, and the same URL maps to the same code. The problem is collisions, which the birthday bound makes concrete. With N = 3.52 × 10^12 codes and n = 6 × 10^9 stored links, the expected number of colliding pairs is about n² / 2N = 3.6 × 10^19 / 7.04 × 10^12, roughly 5 million. The last inserted link has a n/N = 0.17 percent chance of hitting an existing code.

So you need a unique constraint and a retry loop that re-hashes with a salt, adding a round trip to the write path. Deterministic dedupe also breaks down when two customers shorten the same URL and want separate analytics and ownership; you then salt per user and lose the dedupe anyway.

## Option 3: pre-allocated key ranges

Each app instance leases a block of integers (say 10,000) from a central allocator in one atomic operation, then hands them out from memory. The central store is touched once per 10,000 links. A crashed instance wastes its unused block, which is harmless in a keyspace this large.

The pick here is Option 3 with a scramble: lease ranges, and pass each integer through a bijective permutation before encoding. Multiplying modulo 62^7 by a number coprime with 62^7 (which is 2^7 × 31^7) maps every integer to a distinct code and spreads neighbours far apart, which also keeps hash-sharded inserts evenly distributed.

const ALPHABET = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz";
const WIDTH = 7;
export const KEYSPACE = 62n ** BigInt(WIDTH); // 3,521,614,606,208

// Multiplier is coprime with 62^7, so id -> code is a bijection on [0, 62^7).
const MULTIPLIER = 2_176_477_521_739n;
const OFFSET = 123_456_789n;

export function encodeBase62(n: bigint, width = WIDTH): string {
  let out = "";
  for (let v = n; v > 0n; v /= 62n) out = ALPHABET[Number(v % 62n)] + out;
  return out.padStart(width, ALPHABET[0]);
}

export function codeFor(id: bigint): string {
  if (id < 0n || id >= KEYSPACE) throw new RangeError("id outside the 7-character keyspace");
  return encodeBase62((id * MULTIPLIER + OFFSET) % KEYSPACE);
}

// Each app instance leases a block of ids (for example from a Postgres sequence
// declared with INCREMENT BY 10000) and hands them out from memory.
export class RangeAllocator {
  private next = 0n;
  private end = 0n;
  private pending: Promise<void> | undefined;

  constructor(
    private readonly leaseBlockStart: () => Promise<bigint>,
    private readonly blockSize: bigint,
  ) {}

  async nextId(): Promise<bigint> {
    while (this.next === this.end) {
      this.pending ??= this.refill().finally(() => { this.pending = undefined; });
      await this.pending;
    }
    return this.next++;
  }

  private async refill(): Promise<void> {
    const start = await this.leaseBlockStart();
    this.next = start;
    this.end = start + this.blockSize;
  }
}

A test harness confirmed that 2 million consecutive ids produce 2 million distinct seven-character codes, out-of-range ids throw, and 1,000 concurrent callers with a block size of 100 get 1,000 unique ids using exactly 10 leases. leaseBlockStart can be backed by a PostgreSQL sequence created with INCREMENT BY 10000, so each nextval returns the start of the next block (sequence docs). Sequences are never rolled back, so gaps appear, which is fine here.

One caveat: the multiplicative scramble is obfuscation, not secrecy. Someone who collects a few codes can recover the pattern. If links must be unguessable, as with private documents, use long random tokens and accept the longer URL.

Custom aliases bypass the generator and need an atomic insert-if-absent: a unique constraint in SQL, or a conditional put such as DynamoDB's attribute_not_exists (condition expressions). Keep aliases in a different length or character class from generated codes so the two never collide.

The figure below shows the idea: the id space is one long sequence, and each instance owns a separate slab of it.

# Data model and store choice

The access pattern is a point lookup by code, with no joins and mostly immutable rows.

CREATE TABLE links (
  code        text PRIMARY KEY,
  long_url    text        NOT NULL,
  owner_id    bigint      NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT now(),
  expires_at  timestamptz,
  is_active   boolean     NOT NULL DEFAULT true
);
CREATE INDEX links_owner_idx ON links (owner_id, created_at DESC);

The recommendation: a relational database with read replicas and a cache in front, until measurement says otherwise. Three terabytes and 200 writes per second suit a well-provisioned primary, alias uniqueness comes free, and most teams already operate one. A partitioned key-value store (DynamoDB, Cassandra or similar) is reasonable from day one if you want horizontal scale without a resharding project, since the workload is a pure key lookup. It makes the owner index and ad hoc queries harder, so it is a trade-off, not a verdict.

# The read path

A redirect passes through a content delivery network (CDN) or edge, a load balancer and the redirect service. The service then tries an in-process least recently used (LRU) cache, Redis and a read replica, with short-lived entries governed by a time to live (TTL).

client -> CDN / edge -> load balancer -> redirect service
                                           |-> in-process LRU (tiny, ~seconds TTL)
                                           |-> Redis (cache-aside)
                                           |-> database read replica

Each layer is filled on the way back (cache-aside). Four details matter:

• →Cache negative results with a short TTL, so scans for unknown codes do not hit the database.
• →Bound the TTL by the link's expiry, so an expired link is never served from cache.
• →Coalesce misses. A viral link whose entry expires can send thousands of simultaneous reads to the database, the stampede covered in the caching guide.
• →Write the cache on create. A new link read from a lagging replica returns 404, and that negative result may be cached. Writing the mapping at creation avoids both.

The figure below shows how the layers divide the traffic.

# Redirect semantics: 301 or 302

RFC 9110 defines 301 (Moved Permanently) as heuristically cacheable, so browsers and intermediaries may remember it even without explicit freshness headers. A 302 (Found) is cacheable only if the response carries explicit freshness information.

With 301, a repeat visitor's browser may skip your service entirely, so analytics undercount and a takedown does not reach browsers that cached the redirect. With 302 you see every click and can change the destination, at the price of more traffic.

Default to 302 for any product with analytics, editable targets, expiry or abuse takedown, which is almost every shortener. Use 301 only for permanent vanity links where passing along search signals is the point. For a little controlled browser caching, send a 302 with a short explicit Cache-Control such as private, max-age=60. The same logic applies to a CDN: caching redirects at the edge cuts origin load but hides clicks from your counters, unless the edge emits its own click events.

# Analytics pipeline

Never write to the database on the redirect path. The handler should answer first and record afterwards.

1. •The redirect service emits a small event (code, timestamp, country derived from IP, user-agent class, referrer) into a durable log or queue, using a bounded in-memory buffer so a slow queue cannot slow a redirect. If the buffer fills, drop events. Redirect latency wins over completeness.
2. •A stream processor filters obvious bots and link-preview fetchers, then aggregates counts per code per minute.
3. •Aggregates go to an analytics store (a columnar or time-series database) that serves the stats API. Raw events go to cheap object storage for reprocessing.

Assuming a 200-byte event, 10B / 30 ≈ 333 million clicks per day is about 67 GB of raw events daily, roughly 24 TB per year. That dwarfs the 3 TB link table, so analytics storage is usually the biggest line item. Treat IP addresses as personal data: truncate or hash them and set a retention period.

# Abuse and expiry

Shorteners attract phishers because they hide the destination.

• →Scan on create and rescan later. Check destinations against threat lists. Google's Safe Browsing lists are available for non-commercial use, and commercial services typically use a paid reputation API such as Google Web Risk. A link that was clean on Monday can be malicious on Friday.
• →Make takedown effective. Disabling a link must beat your caches: delete the cache key and keep TTLs and Cache-Control short, so a takedown propagates in minutes.
• →Rate-limit creation per API key and IP, require authentication for bulk creation, and consider an interstitial warning for flagged destinations.
• →Expiry is enforced at read time against expires_at and cleaned up by a background job that deletes expired rows in small batches. Never reuse expired codes: an old printed link that suddenly points somewhere new is a hijack waiting to happen.

# Sharding and replication

At the assumed scale one primary and several replicas are enough.

• →Replication: asynchronous replicas serve reads. Accept replication lag, with the create-time cache write described above to cover read-after-create.
• →Sharding: partition by a hash of the code. Because codes are scrambled, inserts spread evenly. Sequential codes with range sharding would send every insert to the newest shard.
• →Multi-region: serve reads from regional replicas or caches, send creates to a home region, and give each region its own block range so ids never collide. The surrounding infrastructure patterns are in the AWS multi-AZ architecture guide.

# Failure modes

Measure redirect p50 and p99, cache hit ratio, replica lag, queue depth and dropped analytics events; the Prometheus monitoring guide covers the tooling.

# What changes at 100 times the scale

Multiply the assumptions by 100: about 3,900 writes per second on average (about 20,000 at peak), 386,000 reads per second on average (about 1.9 million at peak), and 300 TB of link records after five years.

• →Edge-first redirects. Resolve at the edge from a replicated key-value store or a regional cache, so most redirects never cross an ocean. Click events are emitted from the edge.
• →Partitioned key-value store as the system of record, since a single relational primary no longer fits 300 TB or 20,000 writes per second.
• →Key length. 600 billion links fill about 17 percent of 62^7 = 3.5 trillion codes. That still fits, but plan eight characters (62^8 ≈ 2.18 × 10^14) before headroom gets thin.
• →Tiered storage. Many links likely stop being clicked after their first weeks; measure the access distribution, then move cold links to cheaper storage.
• →Analytics. Raw events reach about 2.4 PB per year (24 TB × 100), so pre-aggregate at the edge or sample, and keep raw events briefly.
• →Cost becomes the constraint. Egress, cache memory and analytics storage dominate the bill, which is where the cloud cost optimization guide becomes part of the design.

# Key takeaways

• →Estimate first: 100 million links a month is read-bound (about 19,300 peak redirects per second) and small in storage (about 3 TB over five years).
• →Seven base62 characters give 3.5 trillion codes, ample for billions of links.
• →Hashing URLs gives millions of expected collisions at billions of links; counters need coordination; leased ranges with a bijective scramble avoid both. The scramble hides sequence but is not secrecy, so use random tokens for private links.
• →Return 302 by default to count clicks, change targets and take down abuse; reserve 301 for permanent vanity links.
• →Serve redirects from layered caches with negative caching, coalesced misses and expiry-aware TTLs, and keep analytics off the database path.
• →Plan for abuse on day one: scan, rescan, rate-limit, and make takedown beat the cache.
• →At 100 times the scale, move to the edge, a partitioned store and pre-aggregated analytics; cost becomes the constraint.

# Further reading

• →RFC 9110: HTTP Semantics, including the 301, 302, 307 and 308 redirect status codes.
• →RFC 9111: HTTP Caching, freshness and cacheability for redirect responses.
• →PostgreSQL: CREATE SEQUENCE, INCREMENT BY and gaps.
• →Amazon DynamoDB: condition expressions, conditional writes for insert-if-absent.
• →Google Safe Browsing, checking URLs against unsafe-resource lists.