18 min read Foundations Lesson 08 of 12
Course 02 · Part 3 — Memory, Storage & I/O

Files & File Systems

Where and How Is Data Actually Stored?

When you save vacation.jpg on your computer, does a file with that name actually sit inside your SSD? Are files and folders real physical objects, or are they a masterfully engineered abstraction of the operating system?

Is vacation.jpg Actually Inside Your SSD?

You take a photo with your camera and save it on your computer. Opening File Explorer, you see:

Photos/
└── vacation.jpg

From the surface, this feels completely natural. But if you peer deep into the microscopic realm of silicon, a surprising truth emerges: Inside your SSD, there is no file named “vacation.jpg”!

An SSD or hard drive understands neither English names, nor folders, nor photos. At the hardware level, everything is ultimately billions of electrical charges trapped in silicon cells—raw Digital Bits (0s and 1s). So what is a file? Where do folders live? How does the computer locate and reconstruct our data when we double-click? Let's dismantle the entire storage architecture.

1. What Is a File, Really?

Computer science provides a precise definition:

“A file is a named, organized collection of data that the operating system and user applications can treat as a single logical unit.”

Every day, we interact with various file types:

photo.jpg Compressed image pixel data
movie.mp4 Video frames and audio streams
document.pdf Text, vector shapes, and fonts
program.exe Machine code instructions for CPU
database.db Structured B-tree records
hello.txt UTF-8 encoded character bytes

Regardless of what type of file it is, under the hood lies raw binary data:

photo.jpg: 11111111 11011000 11111111 11100000... (0xFF 0xD8 JPEG Magic Bytes)
hello.txt: 01001000 01100101 01101100 01101100... ('H', 'e', 'l', 'l')

Therefore: A file is a human-meaningful abstraction; at the hardware tier, it is fundamentally a sequence of bits and bytes.

2. Filename, Extension, and Underlying Bytes

Is the Filename Part of the File Data?

When we see photo.jpg, we might assume the letters of the filename are glued directly to the image pixels. In reality: The filename is not part of the file’s payload body!

The file system maintains filenames and their locations in an entirely separate metadata directory table:

Filename: "photo.jpg"
↓ Mapped in directory entry table ↓
File System Metadata (Inode / MFT Record)
↓ Points to allocated storage blocks ↓
Actual File Data Blocks (Raw Bytes on SSD)

What Does a File Extension Actually Do?

Suffixes like .jpg, .pdf, .mp4, or .exe are called File Extensions. They provide hints to the operating system and user applications about which program should handle the file.

Warning: An extension does not create a file format!

If you rename a plain text file from notes.txt to notes.mp4, it does not miraculously become a video. Opening it in a media player will trigger an error because the bytes do not comply with the MP4 container specification. A file’s actual format is governed by its initial Magic Bytes / Signature and its internal binary specification.

Magic Numbers / Signatures

Most formats begin with unique signature bytes. JPEGs start with FF D8 FF, PDFs begin with %PDF-, and Linux ELF executables start with 0x7F 'E' 'L' 'F'.

Internal Container Structure

An MP4 file internally divides data into hierarchical "atoms" or "boxes", cleanly separating metadata (moov), video bitstreams (H.264/AV1), and audio tracks (AAC).

3. Folders, Paths, and the Hierarchical File System

If a storage drive held hundreds of thousands of files in a single flat directory without organization, finding anything would be chaotic. To organize files meaningfully, operating systems introduce Folders or Directories.

Nested Folders: The Tree Hierarchy

Computers structure file systems as an inverted tree (Tree Data Structure). At the very summit sits the Root, branching out into directories and subdirectories, with files acting as the leaves.

Linux / macOS (Single Unified Root `/`) /
├── bin/
├── etc/
└── home/
    └── salim/
        ├── Documents/
        └── Photos/
            └── vacation.jpg
Windows (Drive Letters `C:\`, `D:\`) C:\
├── Program Files\
├── Windows\
└── Users\
    └── Salim\
        ├── Documents\
        └── Photos\
            └── vacation.jpg

What Is a Path?

The complete sequence of steps from the root down to a target file is known as its Path (e.g., /home/salim/Photos/vacation.jpg). It acts as the unambiguous global address of that file within the file system hierarchy.

4. What Is a File System and Where Does It Live?

The specialized system of data structures and algorithms that transforms raw storage sectors into navigable files, folders, and metadata is called the File System.

NTFS Default on Windows; features journaling, ACL permissions, and transparent compression.
ext4 The workhorse of Linux; inode-based, reliable, highly scalable, and fast.
APFS Apple's modern file system; optimized for SSD flash memory with instant file cloning.
exFAT / FAT32 Universal format for USB flash drives and SD cards; readable across Windows, macOS, and Linux.

Does an SSD Understand File Systems? The Hardware-Software Boundary

No! This is a pivotal architectural separation. SSD hardware knows only blocks, pages, and flash memory cells. The file system is a software construct managed entirely by the operating system kernel:

Application (Photoshop, VS Code, Web Browser)
↓ System Call (open, read, write) ↓
Operating System Kernel (VFS — Virtual File System)
↓ File System Logic (NTFS, ext4, APFS) ↓
Block I/O Layer & Storage Device Driver (NVMe / SATA Driver)
↓ LBA Commands (Read Sector #409600) ↓
SSD Microcontroller (FTL — Flash Translation Layer)
↓ Voltage Sensing on Floating Gates ↓
NAND Flash Silicon Memory Cells

Internal Physical Layout on Disk

How does a file system lay out its own tracking structures alongside your data on disk?

1. Superblock / Partition Boot Record: Records total volume capacity, block size, file system type, and counts of allocated/free blocks.
2. Inode Table / MFT (Master File Table): Catalogs file metadata (size, owner, permissions, timestamps, and data block pointers).
3. Directory Entries Table: Maps directory names to their contained files, pairing human filenames with their respective inode numbers.
4. Data Blocks (Payload Area): The raw storage clusters where your image pixels, video frames, code, and text bytes are written.
5. Free Space Bitmap: A bitmask tracking which blocks are free and available for incoming write requests.

5. File Metadata, Blocks, and Fragmentation

Metadata: Data About Data

If a file's content is its body, Metadata is its passport. In Linux, it is called an Inode, while in Windows NTFS, it is an MFT Record. It maintains:

• File Size: Total byte count and allocated disk space.
• Timestamps: Creation time, last modified (mtime), last accessed (atime).
• Permissions & Ownership: User UID, Group GID, read/write/execute flags.
• Block Pointers / Extents: Pointers tracking exactly which storage blocks hold the file's data.

What Is a Block? Why Allocate in Blocks?

A file system cannot track storage byte by individual byte—the tracking overhead would be astronomical. Instead, disks are divided into fixed chunks called Blocks (typically 4 KB or 4,096 bytes).

A tiny 100-byte text file still consumes an entire 4 KB block on disk (internal slack space). A 12 KB file requires 3 blocks, while a 4 GB movie spans roughly one million blocks!

Fragmentation: Are Blocks Always Stored Contiguously?

No! As drives age and files are repeatedly created, edited, and deleted, free space breaks into scattered pockets. This scattering is known as Fragmentation:

[File A] [File A] [File B] [Free] [File A] [File C] [File A]
Fragmentation Hurter on HDDs

On mechanical hard drives, fragmented blocks forced the physical read/write head to move across platters (Seek Time), drastically tanking performance.

No Mechanical Seek on SSDs

SSDs have zero moving parts. The SSD controller accesses any silicon cell across flash dies instantaneously. Defragmenting an SSD is unnecessary and causes harmful wear.

6. How the Operating System Finds a File (Directory Lookup)

When an application requests /home/salim/Photos/vacation.jpg, how does the OS isolate this exact file among millions of blocks?

Path Traversal Mechanics:

  1. Root Inode Lookup: The OS begins at the fixed root directory / (traditionally Inode #2 in Unix) and scans its directory table for home.
  2. Directory Tree Traversal: Reading the inode of home, it locates salim. Inside salim, it retrieves the inode for Photos.
  3. Target File Resolution: Inside the Photos directory table, it matches vacation.jpg to its inode (e.g., Inode #41920).
  4. Block Pointer Extraction: Inspecting Inode #41920, the kernel extracts the list of disk blocks holding the image data (e.g., Blocks 102, 103, 508, 920...).
Path: /home/salim/Photos/vacation.jpg
↓ Root Inode → 'home' Inode → 'salim' Inode → 'Photos' Inode → 'vacation.jpg' Inode ↓
Physical Data Blocks: [102, 103, 508, 920]

7. File Open, File Read, and File Descriptors

A common myth is that "opening a file reads its entire content into memory." In reality, opening and reading are distinct system operations.

1. File Open (Handshake & Permission Check)

The OS traverses the path, validates security permissions, and establishes an active file tracking entry. No payload data is loaded yet! The OS simply returns an integer handle: the File Descriptor (FD) (or File Handle in Windows).

2. File Read (Data Streaming)

When the program executes read(fd, buffer, 1024), the kernel commands the disk controller to fetch requested blocks, transferring the raw bytes into the application's RAM buffer.

// C Language File Lifecycle:
int fd = open("vacation.jpg", O_RDONLY); // Kernel assigns FD index (#3)
read(fd, buffer, 4096); // 4 KB streamed from disk into memory
close(fd); // Handle and kernel table entry released

In Unix-like systems, every process starts with three default open descriptors: 0 = Standard Input (stdin), 1 = Standard Output (stdout), and 2 = Standard Error (stderr). Any newly opened file typically receives descriptor 3.

8. Permissions, Formats, and Image Decoding

File Permissions & Security

In a multi-user OS, users must be prevented from tampering with others' private files or system binaries. The file system enforces strict Permissions:

-rwxr-xr-- 1 salim users 4194304 Sep 22 00:00 vacation.jpg
rwx (Owner): Read, Write, Execute
r-x (Group): Read and Execute
r-- (Others): Read-only access

How Raw Bytes Turn into Visible Pixels

The file system is solely responsible for serving raw byte sequences to the application. Transforming those bytes into meaningful images is the job of the application's Decoder:

Raw Bytes on SSD: [0xFF, 0xD8, 0xFF, 0xE0, 0x48, 0x91...]
↓ File System Read System Call ↓
RAM Application Buffer
↓ JPEG Decompression Algorithm (Huffman Coding & IDCT) ↓
Uncompressed 2D Pixels: RGB(240, 180, 50), RGB(242, 182, 52)...
↓ GPU Framebuffer & Monitor Display Pipeline ↓
Screen Display (Photons striking your eyes!)

9. What Actually Happens When You Delete or Format?

When you press Delete, does the computer immediately wipe all physical bits across the storage cells?

Delete is not instant destruction — it's an unlinking of metadata!

When a file is deleted, the OS simply removes the filename from the directory table and updates the Free Space Bitmap to mark those blocks as available. The physical data bytes remain completely untouched on disk! This explains why data recovery software can easily resurrect recently deleted files.

SSDs, Garbage Collection, and TRIM

Unlike magnetic hard drives that can overwrite existing sectors in place, NAND flash cells must be completely erased (at the block level) before new bits can be programmed.

To manage this efficiently, the OS issues a TRIM command upon deletion, informing the SSD controller: “These logical blocks are no longer in use.” Later, the SSD's background Garbage Collection routine consolidates valid data and erases stale flash blocks.

Quick Format vs. Secure Wipe

Quick Format (Logical Re-initialization)

Rewrites only the file system partition headers, root directory tables, and allocation bitmasks. The underlying user data blocks remain untouched until overwritten.

Secure Erase / Zero Fill (Physical Overwrite)

Physically writes zeros (0x00) or cryptographic pseudorandom patterns across every single sector of the drive, making forensically reconstructing previous data impossible.

10. SSDs, FTL, and NAND Flash at the Silicon Level

While legacy HDDs used spinning platters and magnetic needles, modern solid-state drives (SSDs) are driven by high-speed microcontrollers and non-volatile NAND Flash Memory chips.

FTL — Flash Translation Layer

The OS communicates in Logical Block Addresses (LBAs). But the SSD controller runs sophisticated firmware known as the FTL (Flash Translation Layer):

OS File System Request: LBA #5000
↓ FTL Dynamic Address Translation ↓
Physical Flash Chip #2, Die #1, Block #840, Page #12
↓ Wear Leveling & Bad Block Management ↓
Non-volatile Silicon Charge Trapped

Because NAND flash cells degrade after a certain number of program/erase cycles, the FTL distributes write operations evenly across the entire drive via Wear Leveling, preventing premature failure of any single sector.

NAND Flash Cell Types: How Bits Are Trapped

Connecting back to Course 01's hardware foundation, data in flash cells is stored by trapping electrons inside floating gates or charge traps, where distinct voltage thresholds represent digital bit patterns:

SLC Single-Level Cell 1 bit/cell • Maximum speed & endurance
MLC Multi-Level Cell 2 bits/cell (4 voltage states)
TLC Triple-Level Cell 3 bits/cell (8 voltage states)
QLC Quad-Level Cell 4 bits/cell (16 voltage states)

11. The Complete Journey of a Photo: From Camera Lens to Your Eyes

Trace the full technological continuum from capturing a photo to storing it on disk and rendering it back onto your screen:

📸 Saving: Photons to Silicon
1. Photon Light Waves
↓ Focused through optical lens onto sensor
2. CMOS Sensor (Electrical Signals)
↓ ADC (Analog-to-Digital Conversion)
3. Raw Digital Pixels (RGB)
↓ Image Signal Processor (ISP)
4. JPEG / HEIF Compression
↓ Application System Call (write)
5. OS Kernel & Virtual File System
↓ File System Block Allocation
6. Storage Device Driver (NVMe)
↓ SSD Controller & FTL
7. NAND Flash Floating Gate Charge
🖼️ Opening: Silicon to Human Eyes
1. NAND Flash Trapped Charge
↓ Voltage Sensing & Read Operations
2. SSD Controller / DMA Transfer
↓ Storage Driver Interrupt & Buffering
3. Kernel Page Cache & VFS
↓ Application read() buffer fill
4. Photo Viewer Process Memory
↓ Image Decoder (JPEG Huffman / IDCT)
5. Uncompressed Pixel Array (RGBA)
↓ GPU Framebuffer & Display Pipeline
6. Monitor Pixels & Your Retina

12. File Copy vs. File Rename: The Architectural Difference

Have you ever wondered why renaming a 10 GB file finishes in a millisecond, while copying that same file takes minutes?

File Rename (Instantaneous Metadata Tweak)

Renaming within the same file system volume does not touch or move a single byte of payload data!

Directory Entry: "old.mp4" ──► "new.mp4"
Data Blocks: 100% untouched!
File Copy (Heavy I/O Workload)

Copying forces the file system to allocate completely new free blocks on disk:

1. Read all 10 GB data from source blocks
2. Stream data through RAM buffers
3. Write new 10 GB payload to destination blocks

13. Journaling and File Systems vs. Databases

What Is Journaling? Crash Consistency and Recovery

Imagine updating file metadata when a sudden power outage strikes. If only half the metadata records are written, the entire volume can become corrupt.

Modern file systems (such as NTFS and ext4) solve this via Journaling. Before altering core file system structures, the operation is logged into a dedicated circular journal buffer:

1. Write Intent to Journal ──► 2. Commit Journal Record ──► 3. Write Metadata ──► 4. Checkpoint

If power cuts out, upon reboot the OS inspects the journal and safely replays incomplete actions (roll-forward) or undoes partial writes (roll-back), guaranteeing volume integrity.

File System vs. Database: What Should You Use?

Both systems persist data, but they are built for entirely different abstractions:

Characteristic File System Database (RDBMS / NoSQL)
Data Type Opaque byte streams (images, videos, binaries) Structured records (tables, rows, JSON documents)
Organization Hierarchical paths (folders and files) Relational indexes and schemas (B-Trees, Hash Tables)
Querying Simple path lookup (Open by Name) Expressive queries (SQL: WHERE age > 25 AND active = true)
Concurrency & ACID Coarse file locking Granular multi-user transactions (ACID Guarantees)

In modern web services (like Instagram or YouTube), user profiles, likes, and relational comments reside in a Database, while raw high-resolution video streams and photos are stored in dedicated File / Object Storage (such as Amazon S3 or local distributed file systems).

14. The Grand Abstraction: From Silicon to Screen

When you glance at your desktop, you see a clean icon labeled vacation.jpg. Beneath that simple name lies one of computer science’s most brilliant towers of abstraction:

User Interface: File Icon & Name ("vacation.jpg")
↓ Application System Calls (open, read) ↓
Operating System & VFS (Virtual File System)
↓ File System Metadata & Inode Mapping ↓
File System Logic (NTFS / ext4 / APFS)
↓ Block Device Driver & LBA Commands ↓
SSD Controller & Flash Translation Layer (FTL)
↓ Physical Flash Chips, Dies, Planes & Blocks ↓
NAND Flash Cells (Floating Gates / Charge Traps)
↓ Quantum Physics ↓
Trapped Electrons & Electric Charge (0s and 1s)

From the electricity and transistors explored in Course 01 to the modern file systems of today, all of digital computing is anchored upon this harmonious stack of Abstractions.

🧠 Key Takeaways

  1. A file is a logical abstraction: A named, structured collection of bytes; physical storage knows only electrical charges and bits.
  2. Filenames are not part of the file body: Names live in separate directory tables, pointing to metadata and storage blocks.
  3. Extensions do not create formats: The file’s internal binary structure and magic bytes dictate its true format, not its filename extension.
  4. Folders form a tree hierarchy: Organized starting from a root (/ or C:\) down to files via paths.
  5. SSDs do not understand files: The file system (NTFS, ext4, APFS) creates the illusion of files and folders over raw storage blocks.
  6. Metadata differs from format data: File system metadata tracks where blocks live; format decoders determine how bytes are rendered.
  7. Files are allocated in fixed blocks: Typically 4 KB clusters; fragmentation occurs across disks but carries zero seek penalty on SSDs.
  8. File open and read are distinct operations: Opening verifies security and creates a File Descriptor (FD); reading streams payload bytes into RAM.
  9. Deletion is not instant destruction: The OS merely removes directory entries and frees allocation bits; SSD TRIM cleans stale blocks later.
  10. FTL is the SSD's brain: It translates logical block addresses to physical flash locations and performs wear leveling to maximize drive longevity.
  11. Journaling prevents corruption: Logging operations before modifying file system metadata ensures crash recovery during power failures.
  12. Databases and file systems serve distinct needs: Opaque media stays in file/object storage; relational, queryable data belongs in databases.
🚀 Next Lesson

Lesson 09 — Input, Output & Device Drivers: How Does Software Talk to Hardware?

We have mastered processes, memory, and permanent storage. But a computer is far more than a CPU and an SSD. We have keyboards, mice, monitors, USB devices, sound cards, and network adapters. How does pressing a key on your keyboard turn into a character in your code editor in mere microseconds? How does the operating system converse with this dizzying zoo of external hardware?

“A device driver is the quiet multilingual diplomat standing between the operating system and the quirky engineering of physical hardware.”
Start Lesson 09