Files & Data Persistence: How Programs Read & Write Stored Data

01-02 What is Data Persistence & What is a File?

In computer science, Persistence refers to the characteristic of data that survives the execution lifetime of the process that created it.

What is a File, Exactly?

Simply put, a file is a named collection of bytes stored on physical non-volatile media, referenced by an operating system file system. Every file possesses a name, an extension (.txt, .csv, .json), a path, byte dimensions, and operating system permission flags.

03-06 File Opening, Reading & The Safe with open() Pattern

Python interacts with files through the fundamental built-in open() function:

The Pythonic Standard: Context Managers (with open())

If an exception crashes your script between open() and close(), the file remains locked in limbo. Python solves this via Context Managers (with blocks), ensuring deterministic closure even under fatal exceptions:

07-11 File Modes (`r`, `w`, `a`, `x`) & Newline (`\n`) Caveats

How the OS prepares a file is controlled by the mode argument passed to open():

Explicit Newlines (\n)

Unlike print(), which automatically appends a newline character, file.write() writes verbatim text. You must explicitly supply \n to create line breaks:

12-15 Multiple Ways to Read Files: Memory Ingestion vs Streaming

Python provides four primary access patterns depending on file magnitude and memory constraints:

content = file.read()       # Whole file as a single str
chunk = file.read(100)      # Reads precisely 100 characters

with open("huge_access.log", "r", encoding="utf-8") as file:
    for line in file:
        print(line.strip())  # Discards previous line from RAM as next is fetched

header = file.readline()    # Consumes top line (e.g. CSV headers)
row1 = file.readline()      # Consumes next row

lines = file.readlines()
# Result: ["Apple\n", "Banana\n", "Mango\n"]

Bridging File Streams to String Processing

Applying the string methods learned in Lesson 11 directly transforms raw disk data into structured program structures:

16-18 Persisting User State & File Paths (Relative vs. Absolute)

Saving user interactions to disk allows our software to recognize returning users across application sessions:

Relative Paths vs. Absolute Paths

Specifying where a file lives on the storage tree follows two distinct philosophies:

19-21 UTF-8 Encoding, Text vs. Binary Mode & The Byte Pipeline

At the hardware layer, storage drives only record binary pulses (0s and 1s). Always pass explicit encoding="utf-8" to guarantee consistent rendering across platforms:

Text Mode vs. Binary Mode (`rb`, `wb`)

Binary assets—such as images (.png, .jpg), audio (.mp3), compressed archives (.zip), or executables—do not represent printable characters. Opening them with "rb" or "wb" operates on raw bytes:

22-27 Structured Data Persistence: CSV vs. JSON

Real software requires structured tabular or hierarchical data formats rather than freeform text blobs:

Architectural Comparison: CSV vs. JSON

28-32 Error Caveats, OS Permissions & Files vs. Databases

A cardinal rule of systems programming: External physical resources can never be assumed to exist or succeed without fail.

Are Files Databases?

No! While flat files work wonderfully for small local stores and configs, large-scale software requires dedicated Database Management Systems (DBMS):

33-36 Real-World Telemetry Logging & A Complete Persistent Notes App

Production servers record append-only operational events to log files (app.log) to audit transactions and investigate incidents:

A Complete Persistent Notes Application

A complete script showing state persistence: receiving user input, appending it to disk, and reloading historical entries:

37-42 RAM vs. Storage, Functional Modularity & The Complete Data Flow

Keep file operations cleanly modularized within reusable functions rather than scattering open() calls across your business logic:

True mastery comes through active synthesis. Implement each solution independently, then expand the panel to review reference implementations:

name = input("Enter your name: ")
with open("name.txt", "w", encoding="utf-8") as f:
    f.write(name)

with open("name.txt", "r", encoding="utf-8") as f:
    print(f"Loaded from disk: {f.read()}")

note = input("Write note: ")
with open("journal.txt", "a", encoding="utf-8") as f:
    f.write(note + "\n")
print("Note appended successfully!")

with open("names.txt", "r", encoding="utf-8") as f:
    for idx, line in enumerate(f, start=1):
        print(f"Entry {idx}: {line.strip()}")

with open("manuscript.txt", "r", encoding="utf-8") as f:
    text = f.read()
    lines = text.splitlines()
    words = text.split()
    chars = len(text)

print(f"Lines: {len(lines)}, Words: {len(words)}, Characters: {chars}")

target = input("Search query: ").lower()
with open("document.txt", "r", encoding="utf-8") as f:
    for line_no, line in enumerate(f, start=1):
        if target in line.lower():
            print(f"Line {line_no}: {line.strip()}")

import csv
with open("students.csv", "r", encoding="utf-8") as f:
    reader = csv.reader(f)
    header = next(reader)
    for row in reader:
        print(f"Student: {row[0]}, Score: {row[1]}, Grade: {row[2]}")

import json
developer = {"name": "Salim", "role": "Engineer", "skills": ["Python", "SQL"]}

with open("dev.json", "w", encoding="utf-8") as f:
    json.dump(developer, f, indent=2)

with open("dev.json", "r", encoding="utf-8") as f:
    loaded = json.load(f)
    print(f"Skills: {', '.join(loaded['skills'])}")

count = 0
try:
    with open("counter.txt", "r") as f:
        count = int(f.read().strip())
except FileNotFoundError:
    count = 0

count += 1
with open("counter.txt", "w") as f:
    f.write(str(count))

print(f"Program has executed {count} time(s) across reboots!")