15 min read Part 1 — Foundations of Problem Solving Lesson 01 of 12
Course 06 · Part 1 — Foundations of Problem Solving

What is an Algorithm? What is Problem Solving?

The Art of Problem Solving, Step-by-Step Logic, and Algorithmic Thinking

In our previous course on Data Structures, we explored how data is organized and stored in memory. But storing data alone does not solve problems. To transform raw data into answers and achieve desired goals, we need a precise recipe and procedure. That step-by-step procedure is called an Algorithm, and cultivating the mindset to design it is Problem Solving.

What Will We Learn in This Lesson?

By the end of this foundational lesson, you will clearly understand:

What an algorithm is and why it exists
The relationship between Problem Solving and Algorithms
The fundamental distinction between an Algorithm and a Program
Why multiple algorithms can solve the exact same problem
The 10-step systematic Problem-Solving Framework
Real-world impacts on software systems, performance, and cost

01 What Do We Actually Tell a Computer to Do?

Imagine handing a slip of paper with numbers written on it to a friend and asking:

“Could you find the largest number on this paper for me?”

Here, you posed a problem. In a split second, your friend scans the numbers visually and answers: “The largest number is 95.”

However, if you simply tell a computer “find the largest number”, it has no clue what to do. Computers lack human intuition, common sense, and visual gestalt. To make a machine solve this problem, you must prescribe every atomic, unambiguous step:

1
Take Input: Receive the list of numbers.
2
Initial Assumption: Assume the very first number is currently the largest (largest).
3
Compare: Inspect the next number in the list and compare it with the current largest.
4
Conditional Update: If this next number is greater than largest, update largest to hold this new value.
5
Iterate: Repeat this comparison for every remaining element in the list.
6
Output: After reaching the end of the list, declare the value remaining in largest as the final answer.

This structured sequence of logical, deterministic steps is what we call an Algorithm.

02 What is an Algorithm?

In straightforward terms:

“An Algorithm is an ordered, well-defined, and unambiguous sequence of finite computational steps that solves a specific problem or produces a desired output from given inputs.”

In computer science, a well-formed algorithm possesses six essential characteristics:

1. Well-Defined Problem

It is tailored to solve an explicit, clearly specified problem without vagueness.

2. Step-by-Step Structure

Each instruction connects logically to the preceding one and executes in a distinct sequence.

3. Unambiguous

Every step has a single, definitive meaning leaving zero room for subjective interpretation.

4. Defined Input

It accepts zero or more well-specified inputs from an external source or caller.

5. Meaningful Output

Upon execution, it produces at least one tangible result or state transformation.

6. Finiteness

The algorithm must terminate after a finite number of steps; it can never loop forever.

Algorithms are not restricted to computers. Human civilization and daily routines have relied on algorithmic procedures long before computers existed.

03 Everyday Life Algorithms

Consider a familiar daily routine: preparing a cup of milk tea.

Real-World Recipe: The Milk Tea Algorithm

A step-by-step physical procedure
Input: Water (1 cup), Tea leaves (1 tsp), Sugar (1 tsp), Milk (1/2 cup)
  1. Pour 1 cup of water into a kettle and turn on the heat.
  2. Once water begins boiling, add 1 tsp of tea leaves.
  3. Simmer for 2 minutes to extract the liquor.
  4. Pour in 1/2 cup of milk and stir thoroughly.
  5. Add 1 tsp of sugar and bring to a gentle boil.
  6. Turn off heat and strain the tea into a cup.
  7. Serve hot.
Output: 1 cup of freshly brewed hot milk tea

What happens if you invert the sequence—pouring cold milk and raw tea leaves into an empty cup without boiling water? You won't get tea! The exact order and validity of steps are indispensable to producing the correct outcome.

04 Are Algorithm and Program the Same Thing?

No. While closely related, they represent two entirely distinct layers in computer science:

Algorithm (Logic & Blueprint)

The abstract plan and logical strategy to solve the problem. It is completely independent of any programming language. You can draft it in plain English, on paper, or as a flowchart.

Program (Implementation & Code)

The translation of that blueprint into executable syntax using a concrete programming language (such as Python, Go, or C++) so a computer CPU can run it.

A Concrete Code Illustration:

Problem: “Find the maximum number in a list.”

# Algorithm Logic:
1. Assign the first list item to largest.
2. For each subsequent number in the list:
3. If number > largest, update largest = number.
4. Output largest.

Now we implement this abstract algorithm in Python:

# Program: Python Implementation
numbers = [8, 3, 15, 6, 10]

# Step 1: Initialize largest with the first element
largest = numbers[0]

# Step 2 & 3: Iterate and conditionally update
for number in numbers:
    if number > largest:
        largest = number

# Step 4: Output the result
print(f"The largest number is: {largest}")

Here, the Python code is the Program. The reasoning and steps behind it are the Algorithm. If tomorrow you write this in Rust, Java, or C#, the syntax will change, but the underlying algorithm remains identical!

Interactive Visual Simulator

Experience Algorithms in Motion

Step through the Max Finding algorithm on an array. Watch how each element is compared against the tracked largest value:

Current Largest: 8
Execution Step: Step 1 / 8
Click "Next Step" to start. We initialize the first element (8) as our tentative largest.

05 Multiple Algorithms for One Problem

One of the most profound realizations in algorithmic thinking is: rarely is there only one way to solve a given computing problem.

Suppose you are given a dataset containing 1,000,000 numbers and asked to find if the number 452,873 exists.

Approach 1 — Linear Search Slow (up to 1,000,000 checks)

Check the first number, then the second, then the third... In the worst case, you must examine every single one of the 1,000,000 elements.

Approach 2 — Binary Search Ultra-Fast (at most 20 checks!)

If the array is already sorted, check the midpoint. If our target is larger, we instantly discard the entire lower half (500,000 items). Next step discards 250,000 items, and so on!

Conclusion: Both algorithms yield the exact same answer. Yet one takes up to a million operations while the other takes 20. This leads directly to the core question of computer science: “Which solution is superior, and why?”

06 Why Are Algorithms Crucial?

Imagine two distinct algorithms that solve the exact same business task.

Algorithm A 1 Second

Instant responsiveness and happy users

Algorithm B 10 Minutes

Users abandon the product; timeouts occur

Both solutions are technically “correct”. But in the real world, a 10-minute operation renders software unusable. When your user base scales from 1,000 to 10,000,000, inefficient algorithms crash servers. A superior algorithm delivers:

  • Time Efficiency: Delivers results in milliseconds even across petabytes of data.
  • Memory Conservation: Uses minimal server RAM, preventing out-of-memory crashes.
  • Scalability: Withstands sudden surges in user traffic without degradation.
  • Cost Reduction: Consumes far fewer CPU cycles, slashing cloud infrastructure costs by thousands of dollars monthly.

This is why professional software engineering moves beyond simply asking “Does the code work?” to demanding: “How efficiently does it scale?”

07 What is Problem Solving Really?

A common pitfall among aspiring developers is jumping straight into a code editor the moment a problem is presented. Seasoned software engineers do the exact opposite: they spend most of their time breaking down and analyzing the problem before typing a single line of code.

“Problem Solving is the comprehensive process of deeply understanding an intricate challenge, breaking it down into manageable subproblems, and designing a verified, optimal, and reliable solution.”

The 10-Step Problem-Solving Framework:

01
Understand the Problem What is the exact goal? Can you restate the requirement in your own words?
02
Identify the Inputs What data is supplied? Integers, strings, arrays, or graph nodes?
03
Clarify the Outputs What exact format, type, or state should the solution return or display?
04
Determine Constraints How large can \(N\) get? Are there time limits (\(1\text{s}\)) or memory limits (\(256\text{MB}\))?
05
Decompose the Problem Divide the monolithic challenge into smaller, independent subproblems.
06
Brainstorm Solutions Consider brute force first, then search for optimization patterns.
07
Formulate the Algorithm Write step-by-step instructions in pseudocode or clear numbered logic.
08
Dry Run & Analyze Trace the steps manually on paper with sample inputs; calculate Big-O time and space.
09
Implement in Code Write clean, idiomatic code in your chosen programming language.
10
Test & Refactor Test edge cases (empty input, single element, negative numbers) and optimize bottlenecks.

08 A Practical Example: Searching Across 10,000 Student Records

Imagine a high school administrative ledger containing 10,000 registered student records. You need to answer:

“Who is the student with ID 7821?”
Method 1: Unsorted Sequential Scan

If the paper records are scattered unsorted, you must start from page 1 and turn every page until you stumble upon ID 7821.

Worst-case comparisons: 10,000 checks!
Method 2: Sorted Binary Search

If the ledger is strictly sorted by student ID, you open directly to the midpoint (ID 5000). Since 7821 > 5000, you instantly eliminate the first 5,000 pages! Repeat with the remaining half...

Worst-case comparisons: at most 14 checks! (\(2^{14} = 16,384 > 10,000\))

10,000 operations compared to 14! This exponential speedup wasn't created by upgrading hardware; it was unlocked purely by picking a superior algorithm.

09 The Relationship Between Data Structures and Algorithms

In 1976, Turing Award laureate Niklaus Wirth titled his landmark textbook with this foundational formula:

Algorithms + Data Structures = Programs

To build great software, you must master both halves of this equation:

Data Structure

How data is stored and organized in memory (The Container). E.g., Array, Linked List, Hash Table, Tree, Graph.

Algorithm

The step-by-step procedure executed over that container to achieve a result (The Action). E.g., Traversing, Sorting, Shortest Path.

They are two sides of the same coin:

  • A Hash Table requires a Hashing Algorithm to distribute keys.
  • A Binary Search Tree relies on a BST Traversal Algorithm.
  • A Graph relies on Dijkstra's or A* Shortest Path Algorithms.

10 Are Algorithms Only for Programming?

Not at all. Algorithmic decision systems actively govern modern civilization every second of every day:

Google Search

PageRank and deep semantic ranking algorithms sort billions of web pages in milliseconds.

Google Maps

Dijkstra and A* pathfinding algorithms compute optimal driving routes considering real-time traffic jams.

YouTube & Netflix

Collaborative filtering algorithms predict what video or movie will captivate you next.

Banking & Fraud Prevention

Anomaly detection algorithms analyze credit card transactions in real-time to flag fraud.

Operating Systems (OS)

Round Robin and priority scheduling algorithms determine which application thread gets CPU time.

Artificial Intelligence (AI)

Modern LLMs and neural nets are powered by Backpropagation, Gradient Descent, and Self-Attention mechanisms.

11 Algorithmic Thinking: Categorizing Problem Archetypes

Exceptional software engineers do not just memorize language APIs; they look at a real-world scenario and immediately recognize its underlying mathematical pattern:

Search Problem? Locating a specific item in a large dataset.
Sorting Problem? Arranging items in a defined order.
Graph Problem? Modeling networks, relations, and dependencies.
Optimization Problem? Maximizing profit or minimizing resource cost.
Scheduling Problem? Fair distribution of tasks over constrained time.
Path Finding? Discovering the shortest route between nodes.

The moment you inspect an ambiguous business requirement and recognize—“This is actually a shortest-path optimization problem!”—you have unlocked true Algorithmic Thinking.

12 A Vital Mindset Shift: Syntax vs. Problem-Solving Logic

Many beginner programmers believe:

“If I memorize every keyword and syntax trick in Python, I will become a great programmer.”

This is an illusion. Syntax is just a hammer. If a carpenter does not understand load bearing, wood grain, or structural balance, owning a gold-plated hammer will not help them build a sturdy table.

Programming languages change with industry trends. Today Python, tomorrow Rust, later Go. But your Problem Solving skills and algorithmic logic remain timeless and portable everywhere.

13 What Will We Learn in This Course?

Our goal in this course is not passive memorization. We will build a complete end-to-end thinking pipeline:

Problem Possible Solutions Algorithm Efficiency (Complexity) Clean Implementation

Across 12 masterclass lessons, we will explore:

Searching & Sorting Mechanics
Time & Space Complexity (Big-O)
Recursion & Divide and Conquer
Greedy Algorithmic Strategy
Dynamic Programming (Memoization & Tabulation)
Graph Algorithms (BFS, DFS, Shortest Paths)
Essential Problem-Solving Patterns
Real-World Systems & Capstone Challenge

Concept Check: Test Your Intuition

Select the best answer to receive immediate feedback and explanation

1. What is the fundamental difference between an Algorithm and a Program?

2. In the worst case, how many comparisons does Binary Search need to locate a record among 10,000 sorted items?

3. Which of the following is NOT an essential characteristic of a well-defined algorithm?

🔑 Key Takeaways

1. Algorithm A finite, ordered, unambiguous sequence of steps designed to solve a specific problem.
2. Program The concrete implementation of an algorithm using a specific programming language.
3. Problem Solving The overarching engineering process of deconstructing challenges into verifiable steps.
4. Algorithm Analysis The formal framework that determines which algorithm scales best under growing datasets.
In our next lesson, we tackle the single most important question in computer science: “If two algorithms achieve the exact same output, which one is better?” Welcome to Time Complexity, Space Complexity, and Big-O Notation!