Failure Is Normal

What You Will Learn

Learn to design with the assumption that servers, networks, databases, and external services can fail.

Core Idea

Distributed systems fail in many ways.

Examples:

Server crashes
Database becomes unavailable
Network request times out
External API returns an error
Queue worker stops
Disk becomes full

A reliable system does not assume failure will never happen. It decides what should happen when failure occurs.

A Simple Mental Model

Client
  |
  v
Application
  |
  +--> Database
  |
  +--> Cache
  |
  +--> Background Worker

Do not worry about building every component at once. System design is mainly about understanding why a component exists, what problem it solves, and how components communicate.

Practical Example

For a payment service:

Order Service -> Payment Provider

If the provider times out, the order service should not blindly create a second payment request.

A safer approach may involve idempotency keys, retries with limits, and a clear pending state.

Key Points

['Every external dependency can fail.', 'Timeouts are better than waiting forever.', 'Retries must be limited and carefully designed.', 'Important operations need idempotency.']

Common Beginner Mistakes

Adding technologies before understanding the problem.
Designing for millions of users when the requirements do not justify it.
Ignoring failure cases.
Treating every system as if it needs the same architecture.
Focusing on diagrams instead of explaining the decisions behind them.

Quick Summary

The goal of this lesson is to build a simple mental model that you can reuse in future designs.

Next, you will learn the difference between strong and eventual consistency.