Lesson 16 — What Happens When Data Becomes Massive? How Cache, Database, Search & Storage Work Together

Suppose you are building a modern e-commerce application.

Your platform has grown to host:

• 10 million registered users
• 2 million catalog products
• 500 million customer orders
• 100 million high-resolution product images
• Hundreds of thousands of verified customer reviews

Now consider this fundamental architectural question:

Would you store all of this data inside a single database?

Technically, many modern database engines are capable of storing diverse data formats—including binary data, text blobs, and key-values.

However, in real-world software architecture, forcing a single storage system to handle every conceivable category of workload is fundamentally suboptimal and prone to catastrophic bottlenecks.

This is because each data access pattern poses an entirely distinct engineering challenge:

Consequently, modern applications almost universally employ a cooperative mesh of specialized storage and data systems.

Consider an online marketplace to see the diverse shapes data takes:

User Information

Name
Email
Password Hash
Role

This is structured data—it fits neatly into rows, columns, unique constraints, and foreign key relationships.

Order & Payment

Order ID
User ID
Purchased Items
Total Amount
Payment Status

This is transactional data—it demands strict atomicity, consistency, and durability. Money must never vanish or be double-counted.

Product Media

laptop_hero.jpg (4.2 MB)
smartphone_review.mp4 (85.0 MB)

These are large binary objects (BLOBs). They consume vast disk space and bandwidth, but need no relational foreign keys or row-level locking.

Search Queries

Here, the system needs full-text tokenization, typo tolerance, synonyms, and BM25 relevance scoring—capabilities standard SQL tables cannot provide efficiently.

Frequently Accessed Hot Data

Instead of executing heavy SQL aggregations on disk 50,000 times a minute, this data should be cached in ultra-fast RAM.

Rather than forcing one system to compromise on all these fronts, engineers deploy specialized systems tuned for each workload.

In the previous lesson, we examined the internal workings of databases.

A database serves as the authoritative, durable source of truth for an application's structured business data:

Users
Products
Orders
Payments & Invoices

Typical questions addressed to a relational database include:

In these exact-match, transactional queries, a relational database (like PostgreSQL or MySQL) is exceptionally capable.

Imagine your homepage features a prominent section: "Popular Courses".

During peak hours, thousands of visitors open the homepage every second.

If every inbound HTTP request hits the primary database directly:

User 1 ─┐
User 2 ─┤
User 3 ─┤
User 4 ─┤  ──> Database (CPU & Disk I/O Overload!)
User 5 ─┤
User N ─┘

The database is repeatedly executing the exact same SQL query to produce the exact same result set. Disk I/O spikes, connection pools exhaust, and queries start timing out.

The database has become unnecessarily overwhelmed by redundant reads.

To solve this architectural bottleneck, we introduce:

A cache is an ultra-fast temporary storage layer (typically stored in RAM) that keeps copies of frequently requested data so future requests can be served substantially faster than fetching from the primary source.

Consider our "Homepage Popular Courses" scenario:

1. First Request:
   Database ──> Fetches Popular Courses ──> Stored into Cache (RAM)

2. Subsequent Requests:
   User ──> Application ──> Cache (RAM < 1ms) ──> Returns Popular Courses
   (The database is never touched!)

Because RAM reads are thousands of times faster than persistent disk lookups, the response is delivered in under a millisecond, and database load drops by up to 95%.

When requested data is found inside the cache memory layer, it is called a Cache Hit.

Incoming Request
       │
       ▼
  Check Cache
       │
   Found! ✅ (CACHE HIT)
       │
       ▼
 Return Immediate Response (<1ms)

Cache hits represent optimal performance: lightning-fast execution, zero disk I/O, and negligible CPU consumption.

When requested data is not present in the cache (either because it was never fetched before or because its time-to-live expired), it is known as a Cache Miss.

Upon a cache miss, the application must fall back to the slower authoritative data source:

Incoming Request
       │
       ▼
  Check Cache
       │
 Not Found! ❌ (CACHE MISS)
       │
       ▼
 Query Database (Disk Scan ~40ms)
       │
       ▼
 Retrieve Data
       │
       ▼
 Write Copy to Cache (for future requests)
       │
       ▼
 Return Response to User

While this initial request incurs database latency, it primes the cache so that subsequent requests experience immediate Cache Hits.

The most ubiquitous caching pattern in modern systems is known as Cache-Aside (Lazy Loading):

                 Incoming Request
                        │
                        ▼
                Check Cache (Redis)
                        │
         ┌──────────────┴──────────────┐
         ▼                             ▼
    [CACHE HIT]                   [CACHE MISS]
         │                             │
    Return Data                        ▼
    Immediately               Query Database (SQL)
                                       │
                                       ▼
                                Store into Cache
                                       │
                                       ▼
                                  Return Data

With Cache-Aside, the application code coordinates both the cache and the database: it attempts to read from cache first, and populates the cache only when a miss occurs.

Here lies the fundamental dilemma of caching.

Suppose an item's price is stored in the database as:

৳85,000

The cache also holds:

৳85,000

Then, an administrator updates the price in the database to:

৳82,000

If the cache is not explicitly alerted, it continues serving ৳85,000 from memory!

Customers see the outdated price. This discrepancy between the cache and the primary database is known as serving Stale Data.

Resolving this is the classic challenge of:

As the famous computer science adage goes: "There are only two hard things in Computer Science: cache invalidation and naming things."

The application must reliably decide:

Several standard strategies exist:

1. TTL (Time-To-Live Expiration)

Each cached entry is stamped with an automatic lifespan (e.g., 60 seconds). Once the time elapses, the cache automatically evicts the key.

2. Explicit Invalidation (Evict on Write)

Whenever data changes in the database, the backend actively executes a cache deletion command (e.g., DEL product:25). The next read will naturally trigger a cache miss and fetch the fresh database value.

3. Key Versioning

Keys include a version identifier (e.g., product:25:v2). When data mutates, the system increments the version pointer, instantly making older cache entries obsolete.

We previously encountered TTL (Time to Live) in Lesson 6 when studying DNS resolvers.

Notice that TTL is not exclusive to DNS—it is a universal foundational concept across all caching systems:

Homepage Popular Courses
TTL = 60 seconds

For 60 seconds, thousands of requests are satisfied at lightning speed from RAM. When the 60 seconds expire, the entry vanishes, prompting the next request to re-query the database for the newest data.

No. Caching is not a single product or isolated server—it is an ubiquitous architectural philosophy implemented at every layer of the request pipeline:

Thus, caching is an end-to-end performance strategy spanning from the user's laptop screen all the way to database storage controllers.

Suppose a user uploads a 100 MB 4K video or a 5 MB raw product image.

Should this 100 MB binary payload be stored directly inside a table column of a relational database?

While most databases provide binary data types (like BLOB or BYTEA), storing massive media files inside relational databases is an anti-pattern:

• Database table size explodes, leading to slow table scans and high storage costs.
• Daily backups (WAL dumps, snapshots) become astronomically large and slow.
• Serving heavy binary bytes exhausts database connections and CPU cycles meant for transactions.

For large media and unstructured file workloads, modern architecture relies on:

Object storage is engineered specifically for:

• Images & Graphics (JPEG, WebP, PNG, SVG)
• Videos & Audio streams (MP4, HLS chunks, MP3)
• Documents & PDFs (invoices, certificates, books)
• Automated database backups & log archives

Popular cloud object storage systems include Amazon S3, Google Cloud Storage, Azure Blob Storage, and Cloudflare R2.

A traditional file system (like NTFS or ext4) uses hierarchical directories, folders, and POSIX file locks.

Object storage operates on a flat namespace. There are no actual physical nested directories on disk—only unique string keys inside a "Bucket":

Bucket: academy-storage
 ├── images/product-1.jpg       (Key: "images/product-1.jpg")
 ├── images/product-2.jpg       (Key: "images/product-2.jpg")
 ├── videos/course-1.mp4        (Key: "videos/course-1.mp4")
 └── documents/lesson-16.pdf    (Key: "documents/lesson-16.pdf")

Every object consists of three essential elements:

• Data Payload: The raw binary bytes of the file itself.
• Metadata: Content-Type, file size, upload timestamp, author ID, and custom tags.
• Object Key: A globally unique identifier string used to address and fetch the object over HTTP.

How do these systems cooperate in real-world production?

The architecture maintains a strict division of labor:

Relational Database:
┌────────────────────────────────────────────────────────┐
│ Product ID: 25                                         │
│ Name: Pro Gaming Laptop 16                             │
│ Price: ৳85,000                                         │
│ Stock: 14                                              │
│ Image Key: "products/25/main.webp"  <── (String ref)  │
└────────────────────────────────────────────────────────┘

Object Storage (Bucket):
┌────────────────────────────────────────────────────────┐
│ Key: "products/25/main.webp"                           │
│ Payload: [3.4 Megabytes of WebP binary image data]     │
└────────────────────────────────────────────────────────┘

Let's trace the concrete sequence of events:

1. Browser makes API call:
   GET /api/products/25

2. Backend queries database and responds with lightweight JSON:
   {
     "id": 25,
     "name": "Pro Gaming Laptop 16",
     "price": 85000,
     "image_url": "https://cdn.example.com/products/25/main.webp"
   }

3. Browser parses JSON, encounters <img src="...">, and sends a separate request:
   Browser ──> CDN Edge / Object Storage ──> Downloads Image Bytes

Notice the critical engineering victory: The primary database never had to stream 3.4 megabytes of image data!

The database remained 100% free to handle high-concurrency transactional queries.

In Lesson 9, we explored how Content Delivery Networks (CDNs) operate globally.

Now let's connect CDN edge nodes with Cloud Object Storage:

                 Global Edge Network
User (Dhaka)  ──>  CDN Edge (Dhaka PoP)  ──>  Origin Object Storage (US-East)

Here is the exact lifecycle of a media request:

1. First User requests product image:
   User ──> CDN Edge (Cache Miss) ──> Object Storage Origin ──> Fetches & Caches Image

2. Next Thousands of Users request the same image:
   User ──> CDN Edge (Cache HIT! ✅ <10ms) ──> Delivers directly from local Edge RAM/SSD
   (Object Storage origin is never accessed again!)

This shields object storage from bandwidth saturation and provides near-instantaneous media rendering to end-users worldwide.

Every component in this tri-part architecture excels at its core competency:

No single system is overwhelmed. Each tier scales independently based on its own traffic dynamics.

Suppose your catalog expands to 10 million products.

A shopper enters the search bar and types:

How would a standard relational database attempt to execute this query?

SELECT * FROM products 
WHERE description LIKE '%wireless%' 
  AND description LIKE '%gaming%' 
  AND description LIKE '%mouse%' 
  AND description LIKE '%black%';

This SQL query causes an unmitigated disaster in production:

• Full Table Scan: Wildcard '%...%' operators cannot use traditional B-Tree indexes. The database must sequentially read all 10 million rows from disk!
• Zero Typo Tolerance: If the user types "wirless", 0 results are returned.
• No Relevance Ranking: SQL cannot score which products are most relevant based on term frequency or semantic context.

To solve this, modern infrastructure introduces a specialized subsystem:

Leading search engine technologies in industry include:

• Elasticsearch & OpenSearch: Distributed, JSON-based search engines built on Apache Lucene.
• Apache Solr: Enterprise search server platform.
• Meilisearch & Algolia: Ultra-fast typo-tolerant, instant search solutions for web frontends.

Before search queries can run at blazing speed, documents must be processed through an Inverted Index pipeline.

Suppose our catalog has the following product titles:

Doc 1: "Black Gaming Wireless Laptop Pro"
Doc 2: "RGB Mechanical Keyboard Black"
Doc 3: "Ergonomic Wireless Gaming Mouse"

The search engine breaks each sentence into individual words (tokens), converts them to lowercase, strips common filler words (stop-words), and builds a reverse lookup table mapping every term to document IDs:

TERM        ──> POSTINGS LIST (Document IDs)
---------------------------------------------
"black"     ──> [Doc 1, Doc 2]
"gaming"    ──> [Doc 1, Doc 3]
"wireless"  ──> [Doc 1, Doc 3]
"laptop"    ──> [Doc 1]
"keyboard"  ──> [Doc 2]
"mouse"     ──> [Doc 3]

When a user searches for "wireless gaming", the search engine does not scan 10 million rows on disk. Instead, it simply takes the intersection of the postings lists for "wireless" and "gaming":

Understanding the distinct strengths of each engine clarifies why they coexist:

In almost all standard architectures, no.

The search index is a read-optimized derivative projection, not the authoritative ledger of your business:

PostgreSQL (Primary Source of Truth)
               │
               ▼  (Data Sync / Indexing Pipeline)
         Elasticsearch (Search-Optimized Projection)

The primary database preserves transactional integrity and audit history. The search engine simply ingests copies of that data to make it searchable.

Suppose a seller modifies a product title in the database:

Old: Wireless Mouse
New: Ultra-Slim Wireless Gaming Mouse

The search engine's inverted index must reflect this change. In modern event-driven architectures, this is handled via an asynchronous pipeline:

1. Admin updates Database ──> SQL UPDATE executed (Committed)
2. Database triggers Event / CDC (Change Data Capture)
3. Event Bus (Kafka / RabbitMQ / SQS) queues the change notification
4. Worker Service ingests event ──> Updates Elasticsearch Inverted Index

Because this pipeline is decoupled and asynchronous, there is a tiny delay (typically a few milliseconds to a couple seconds) before the search index reflects the change.

Suppose an inventory update commits to the primary database right now.

However, the indexing pipeline takes 2 seconds to process the event and refresh the search cluster.

During those 2 seconds:

Database:       Shows New Data immediately
Search Engine:  Shows Previous Data for 2 seconds

This temporary discrepancy is known as Eventual Consistency.

Now we can visualize how all five specialized data systems cooperate in a unified modern web platform:

                           END USERS
                              │
                              ▼
                         CDN / EDGE
                         /                                /                          Static/Media        API Requests
                     │               │
                     ▼               ▼
              Object Storage    Load Balancer
              (S3 / R2)              │
                                     ▼
                               App Clusters
                               /                                        /                                   Cache (Redis)    Primary DB (PostgreSQL)
                                             │
                                             ▼
                                     Search Engine (ES)

This is the conceptual foundation of high-scale enterprise engineering.

When a user visits https://shop.example.com/products/25, observe how the entire architecture collaborates:

Every system played its part perfectly to construct a single screen within milliseconds.

When a user enters "wireless mouse" into the search box:

Frontend:
GET /api/search?q=wireless+mouse
       │
       ▼
Backend API Gateway
       │
       ▼
Search Microservice
       │
       ▼
Elasticsearch Cluster (Inverted Index Match & BM25 Scoring)
       │
       ▼
Returns Top Ranked Product IDs & Snippets:
{
  "total": 420,
  "results": [
    { "id": 25, "name": "Wireless Gaming Mouse Pro", "score": 3.84 },
    { "id": 102, "name": "Ergonomic Wireless Mouse", "score": 2.91 }
  ]
}
       │
       ▼
Browser renders live search results dynamically via JavaScript

Instead of an exhaustive table scan locking the primary database, the dedicated search engine handled the query in 5 milliseconds.

The search engine index is populated directly from the authoritative database via continuous background data pipelines:

PostgreSQL Database ──> CDC / Event Stream ──> Indexing Service ──> Elasticsearch Index

When a user executes a search, the search engine quickly returns the IDs of matching documents. If the frontend requires deeper authoritative details (such as live stock or private customer pricing), the backend hydrates those details from the database or cache.

A common beginner mistake is conflating a cache with a search engine. They solve entirely different computational problems:

                      ┌── Cache Hit  ──> Ultra-Fast Response (<1ms)
                      │
Application ──> Cache
                      │
                      └── Cache Miss ──> Fallback to Database
                                                │
                                                ▼
                                         Disk SQL Fetch (~40ms)
                                                │
                                                ▼
                                         Save to Cache for future

The database provides durability and authority; the cache provides raw performance and throughput.

Database says:       "Here is the product name, price, and file key reference."
Object Storage says: "Here are the actual 5 megabytes of binary image pixels."

Separating references from payloads prevents database bloat and maximizes storage cost-efficiency.

Database:      "Authoritative structured business records and ACID transactions."
Search Engine: "Search-optimized inverted index projection for linguistic discovery."

The search engine serves discovery workloads; the database finalizes actual purchases.

The CDN (Content Delivery Network) sits directly at the edge perimeter between end-users and your core infrastructure:

                 Global Edge Network
User ──> CDN Edge
            ├── Cache Hit  ──> Returns Cached Asset Immediately (<15ms)
            │
            └── Cache Miss ──> Fetches from Origin
                                    ├── Cloud Object Storage (Images, Videos)
                                    └── Application Server (Dynamic APIs)

By terminating connections closer to users and offloading static content, the CDN drastically decreases latency and reduces origin bandwidth costs.

Absolutely not. Caching is not a magic silver bullet for every workload.

Caching highly sensitive or rapidly mutating data introduces severe consistency and security vulnerabilities:

Therefore, caching decisions must be grounded strictly in your application's data consistency tolerance.

No. By its very nature, a cache can drift out of sync and become stale.

Consider an inventory flash-sale scenario:

PostgreSQL Database:  Stock = 0  (Sold Out!)
Redis Cache:          Stock = 1  (Cached 30 seconds ago)

If your checkout service trusts the cache instead of the authoritative database row lock, it will accept orders for out-of-stock items, creating customer disputes and fulfillment failures.

Critical business operations (payments, inventory deduction, identity permissions) must always validate against the authoritative transactional database.

As application read traffic expands, databases scale beyond a single node using Primary-Replica Replication alongside caching:

                              Application Servers
                               /                                (Read Cache) /                 \ (Writes & Transactions)
                             ▼                   ▼
                        Redis Cache         Primary Database (Master)
                                                 │
                                                 ▼ (Asynchronous Replication)
                                            Read Replicas (Slaves)
                                            (Handles heavy read queries)

All write operations (INSERT, UPDATE, DELETE) go to the Primary DB. Non-cached analytical reads are distributed across Read Replicas. Note that asynchronous replication introduces minor replication lag, which must be accounted for in system design.

Imagine your platform reaches 10 billion records.

A single physical database server—regardless of how much RAM or SSD capacity you provision—cannot store the entire dataset or handle the concurrent write throughput.

At this scale, data must be partitioned horizontally across independent database machines:

                               Database Cluster
                                      │
                   ┌──────────────────┼──────────────────┐
                   ▼                  ▼                  ▼
                Shard A            Shard B            Shard C
            (Users 1 - 33M)    (Users 34M - 66M)  (Users 67M - 100M)

This architectural technique is known as:

Sharding solves the physical storage wall, but it introduces substantial architectural complexity:

• Cross-Shard Queries: Joining tables across different physical shards requires network scatter-gather queries, destroying latency.
• Distributed Transactions: ACID transactions across shards require complex two-phase commit (2PC) protocols that degrade performance.
• Resharding & Rebalancing: When a shard becomes a "hot spot" (e.g. a celebrity user), redistributing data across shards is exceptionally difficult.

Every modern web platform must balance three fundamental requirements:

Along with dedicated Search Engines for exploration and Object Storage for rich media, these pillars power today's largest internet applications.

Here is the complete panoramic view of a production-grade multi-tier system:

                                  GLOBAL USERS
                                       │
                                       ▼
                                  CDN / EDGE
                                 /                                          /                                 Static/Media             Dynamic API
                          │                        │
                          ▼                        ▼
                   Object Storage            Load Balancer
                    (S3 / R2)                      │
                          │                        ▼
                          │                 App Server Cluster
                          │                  /                                      │                 /                                        │           Redis Cache      Primary DB (Writes)
                          │                                 │
                          │                         ┌───────┴────────┐
                          │                         ▼                ▼
                          │                     Read Replica    Async Event Bus
                          │                                          │
                          │                                          ▼
                          └───────────────────────────────> Search Cluster (ES)

Every component has a single, well-defined responsibility, ensuring fault isolation and infinite scalability.

Now you can appreciate why modern system architecture diagrams are so rich and layered.

A beginner looks at a screen and says:

A computer systems engineer looks beneath the surface and sees:

DNS ──> CDN ──> TLS Handshake ──> Load Balancer ──> Reverse Proxy ──> App Servers
      ──> Redis Cache ──> Primary DB ──> Read Replica ──> Search Cluster ──> Object Storage

Every single component in that chain solves a distinct, unavoidable physical constraint.

For an early-stage hobby project or classroom assignment:

This is wonderfully simple and completely sufficient.

However, as scale expands, workloads diverge completely:

• Transactional Integrity: Relational Database (PostgreSQL)
• Ultra-Fast Repeated Reads: In-Memory Cache (Redis)
• Large Binary Files: Cloud Object Storage (S3 / R2)
• Full-Text Linguistic Search: Inverted Search Engine (Elasticsearch)
• Global Content Delivery: Distributed Edge CDN (Cloudflare)

A specialized architecture is not over-engineering—it is the direct consequence of physical throughput limits.

Consider how an online computer science platform like Byteion Academy maps its data needs:

Even a focused educational platform is in fact an orchestrated symphony of specialized systems.

Follow a user finding and watching a course:

1. User searches: "python programming"
   Browser ──> DNS ──> TLS ──> CDN ──> Load Balancer ──> App Server ──> Elasticsearch
   Returns ranked course IDs via Inverted Index match in 4ms!

2. User clicks course: "/courses/python-101"
   App checks Redis Cache ──> Miss ──> Queries PostgreSQL DB ──> Caches to Redis
   Returns course JSON payload to browser.

3. Browser requests video lecture thumbnail:
   Browser ──> CDN Edge ──> Cache Hit! Delivered in 6ms (S3 origin untouched).

4. User presses Play:
   Video player streams HLS chunks directly from CDN & S3 Object Storage!

Every system harmonizes to deliver a seamless user experience.

Whenever you evaluate where data belongs, ask these four fundamental questions:

Pause and look back at the grand panoramic journey you have completed:

Physical Hardware (Cables, Fiber, Radio Waves)
       ↓
Network Packets & Routing (IP & BGP)
       ↓
Reliable Transmission (TCP, UDP, Ports)
       ↓
Name Resolution (DNS & Root Servers)
       ↓
Encrypted Communication (TLS 1.3 & HTTPS)
       ↓
Application Protocols (HTTP/1.1, HTTP/2, HTTP/3, WebSockets)
       ↓
Client Execution (Browser Rendering Engine & DOM/CSSOM)
       ↓
API Architecture (REST, JSON, Endpoints)
       ↓
Data Persistence (Relational Databases, SQL, B-Trees, ACID)
       ↓
Unified Multi-Tier Data Layer (Cache, Object Storage, Search Engines, CDN)

You now understand how the disparate disciplines of computer engineering unify to power the modern internet.

Consider this progression:

What happens when millions of users arrive simultaneously?

• Is a single application server sufficient?
• Is a single database cluster enough?
• Can a single data center serve users across Tokyo, London, and New York?
• What happens when a critical server catches fire or a data center loses power?

This brings us to the ultimate discipline of modern systems engineering:

In the next lesson, we tackle the crowning architectural challenge of this course:

We will explore:

Vertical Scaling (Bigger Box)
       ↓
Horizontal Scaling (More Boxes)
       ↓
Stateless Application Servers
       ↓
Load Balancers & Health Checks
       ↓
Database Replication & Read Splitting
       ↓
Automated Failover & High Availability (HA)
       ↓
Geo-Distributed Multi-Region Deployments

You will master the blueprints required to design systems capable of sustaining hundreds of millions of global users without downtime!