System Design: API Design

Master RESTful API design best practices, API versioning strategies, offset vs cursor pagination, idempotency, OpenAPI specifications, webhooks, and GraphQL schema design.

1. RESTful API Design Best Practices & Richardson Maturity Model

Representational State Transfer (REST) is an architectural style for networked applications. High-quality REST APIs adhere to resource-based URL naming, standard HTTP verbs, predictable status codes, and RFC 7807 error envelopes.

Richardson Maturity Model (Level 0 to Level 3 HATEOAS)

System Design

Tracing the progression from POX tunneling to Resource URIs, HTTP verbs, and dynamic hypermedia controls

100%
Loading system design canvas…

Resource Naming & HTTP Verbs

Design predictable, noun-based hierarchical endpoints.

  • Use Plural Nouns: /api/v1/users, /api/v1/orders/{id}/items
  • GET: Retrieve resource (Safe & Idempotent, 200 OK)
  • POST: Create new subordinate resource (Non-Idempotent, 201 Created)
  • PUT: Replace resource completely (Idempotent, 200 OK / 204 No Content)
  • PATCH: Partially modify resource fields (Non-Idempotent / Idempotent)
  • DELETE: Remove resource (Idempotent, 204 No Content)

HTTP Status Codes & RFC 7807 Error Envelope

Communicate exact machine-readable problem states.

  • 2xx Success: 200 OK, 201 Created, 204 No Content
  • 4xx Client Error: 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict, 422 Unprocessable Entity, 429 Too Many Requests
  • 5xx Server Error: 500 Internal Error, 502 Bad Gateway, 503 Service Unavailable
  • RFC 7807: Standard JSON format with type, title, status, and detail properties

2. API Versioning Strategies & Deprecation Lifecycles

APIs evolve over time. When introducing breaking changes (removing fields, altering data types, modifying endpoints), systems must provide seamless versioning without breaking existing integrations.

The 4 Core API Versioning Architectural Strategies

System Design

Comparing URI Path, Query Parameter, Custom Header, and Accept Content Negotiation versioning

100%
Loading system design canvas…

Breaking vs Non-Breaking Changes & Sunset Headers

Guidelines for smooth enterprise API evolution.

  • Non-Breaking (No version bump needed): Adding new optional request fields, adding new response fields, adding new endpoints
  • Breaking (Version bump required): Renaming or deleting fields, changing field types (string to number), changing authentication schemes
  • Deprecation Standards: Return Sunset: Wed, 11 Nov 2026 00:00:00 GMT and Deprecation: @1730000000 HTTP headers to signal upcoming retirement

3. Pagination Techniques: Offset-Based vs Cursor-Based (Keyset)

Returning unbounded dataset queries overburdens database memory and network bandwidth. APIs must paginate lists using either simple offset counters or scalable cursor keysets.

Offset Data Drift Flaw vs Stable Cursor-Based B-Tree Traversal

System Design

Visualizing phantom duplicate reads under real-time concurrent insertions

100%
Rendering diagram…

Offset-Based Pagination (?page=5&limit=20)

SQL: SELECT * FROM items LIMIT 20 OFFSET 100.

  • Pros: Easy to jump to arbitrary page numbers (e.g. Page 12)
  • Cons: O(N) database performance degrades severely on large offsets (OFFSET 1,000,000 scans 1M rows in RAM)
  • Cons: Data Drift bug causes duplicated or missed items during live inserts/deletions

Cursor-Based / Keyset Pagination (?cursor=eyJpZCI...)

SQL: SELECT * FROM items WHERE id > 1042 LIMIT 20.

  • Pros: Constant O(1) indexed B-Tree seek time regardless of dataset size
  • Pros: Immune to data drift duplicates during active insertions
  • Cons: Cannot jump to random pages; supports sequential next/previous traversal only

Production Cursor-Based (Keyset) Pagination in Express.js & Node.js

Efficiently paginate large result sets without the performance cost of offset-based pagination.

4. Idempotent API Design & Safe Mutation Guarantees

Transient network blips often cause clients to retry HTTP POST requests without knowing if the server processed the initial request. Without idempotency guarantees, users get double-charged.

Idempotency Key Atomic Gatekeeper Lifecycle (Stripe Pattern)

System Design

SETNX mutex locking, in-flight deduplication, and cached HTTP response replaying

100%
Rendering diagram…

Idempotency Key Pattern Best Practices

Guaranteeing exactly-once execution semantics for distributed transactions.

  • Client generates unique Idempotency-Key (UUIDv4) in request header
  • Gateway acquires atomic lock in Redis with 120s TTL (SETNX)
  • If key exists and status is COMPLETED, immediately return cached response payload
  • If key exists and status is PROCESSING, return HTTP 409 Conflict

5. API Rate Limiting, Throttling & Standard Quota Headers

Protecting APIs against abuse and noisy neighbor starvation requires enforceable rate limits paired with clear HTTP headers informing consumers of their remaining quota.

Standard IETF / RFC 6585 Rate Limit Headers

Transparent communication of quota state to API clients.

  • RateLimit-Limit: Allowed requests within the current quota time window (e.g. 1000)
  • RateLimit-Remaining: Number of remaining requests permitted before rejection (e.g. 42)
  • RateLimit-Reset: Number of seconds remaining until quota resets (e.g. 18)
  • HTTP 429 Too Many Requests: Returned when quota is exceeded, paired with Retry-After: 30
  • Tiered Throttling: Enforce tiered buckets (Anonymous: 10/min, Free: 100/min, Pro: 5,000/min)

6. Contract-First API Design & OpenAPI / Swagger Specification

Modern engineering teams employ Design-First (Contract-First) workflows: authoring the OpenAPI 3.1 YAML specification before writing a single line of backend implementation.

Contract-First OpenAPI Development Lifecycle & Tooling Pipeline

System Design

Parallel frontend/backend development enabled by mock servers, automated SDK generators, and interactive Swagger portals

100%
Loading system design canvas…

Benefits of Contract-First OpenAPI Specifications

Accelerating cross-team velocity.

  • Parallel Development: Frontend builds against mock servers (Prism) while backend builds DB schemas
  • Automated Client SDK Generation: Generate typed TypeScript, Python, and Go client libraries instantly via openapi-generator
  • Automated Request Validation: Middleware validates incoming payloads against OpenAPI JSON schemas automatically

7. Webhooks vs Polling APIs & Cryptographic Signature Verification

Polling APIs repeatedly request status updates (e.g. checking every 5 seconds if a payment succeeded), wasting bandwidth and compute. Webhooks reverse this communication by pushing real-time HTTP POST notifications when events occur.

Enterprise Webhook Delivery Queue & HMAC-SHA256 Verification Pipeline

System Design

Decoupled asynchronous worker delivery, exponential retry schedules, and anti-replay timestamp verification

100%
Rendering diagram…

Polling vs Webhooks Comparison

Choosing the right communication pattern.

  • Short Polling: High waste (98% of requests return no new data)
  • Long Polling: Holds connection open until data arrives (moderate resource use)
  • Webhooks (Push): Zero waste, instant real-time delivery over standard HTTP POST

Webhook Security & HMAC Verification

Protecting webhook endpoints from spoofing and tampering.

  • HMAC-SHA256 Signatures: Signs payload + timestamp with shared secret
  • Anti-Replay Protection: Rejects events with timestamps older than 5 minutes
  • Constant-Time Comparison: Uses crypto.timingSafeEqual to prevent timing attacks

Webhook HMAC-SHA256 Signature Generator & Anti-Replay Verifier in Node.js

Signs outgoing webhook payloads and verifies incoming ones with a constant-time HMAC check.

8. GraphQL Schema Design & Solving the N+1 Problem with DataLoader

GraphQL allows clients to request exactly the fields they need, eliminating REST over-fetching and under-fetching. However, nested object resolvers naturally cause the notorious N+1 Query Problem unless mitigated by batching utilities like DataLoader.

GraphQL DataLoader Request Batching Sequence (Solving the N+1 Query Problem)

System Design

Collapsing 50 individual child SQL queries into a single batched WHERE IN query

100%
Rendering diagram…

GraphQL Core Concepts (SDL)

Strongly typed query language for APIs.

  • Query: Fetch data without side effects
  • Mutation: Mutate data and return modified entity
  • Subscription: Real-time event streams over WebSockets
  • Apollo Federation: Declarative supergraph router composing multiple subgraphs

DataLoader Batching & Caching

High-performance resolver batching.

  • Event Loop Batching: Gathers all individual IDs requested across a single tick of the Node.js event loop
  • Single Query: Executes 1 SQL query (SELECT * FROM posts WHERE user_id IN (...)) instead of N queries
  • Per-Request Memoization Cache: Prevents querying identical keys multiple times within one request

Knowledge Check

1. What is the major flaw of Offset-Based Pagination on large tables?

2. Which HTTP methods are defined as Idempotent in RFC 7231?

3. How do Webhook consumers prevent Replay Attacks when processing events?

4. How does DataLoader solve the N+1 Problem in GraphQL resolvers?

5. What is the primary advantage of Contract-First OpenAPI specification design?