Lesson 14.14 — Scalars & Vectors

1 Introduction: Can Numbers Alone Express Everything?

Imagine a car traveling down a highway at 60 kilometers per hour. You know its speed is 60 km/h. But which way is the car heading—is it heading North, South, or East? Can you determine its heading from this single number alone?

Clearly not. Knowing only the magnitude of motion tells us nothing about its spatial orientation or direction.

Now suppose the navigation system announces: "The vehicle is traveling at 60 km/h due North." Now you possess two crucial pieces of information simultaneously: the intensity of motion (magnitude) and its precise direction.

Here lie two distinct foundational concepts of mathematics and physics:

• Speed: Fully described by a single scalar numerical value and an associated unit (e.g., 60 km/h).
• Velocity: Demands both a numerical magnitude and an explicit spatial direction (e.g., 60 km/h North).

Scalars and vectors are far from mere high school physics constructs. The entire mathematical backbone of modern Computer Graphics, Game Engines, Robotics, Data Science, Machine Learning, and Large Language Models (LLMs) rests on vectors and Linear Algebra.

2 What is a Scalar?

A Scalar is a mathematical or physical quantity completely specified by its magnitude (numerical value) and appropriate physical unit; no spatial direction is required or meaningful.

In pure mathematics, a scalar is simply a real number ($c \in \mathbb{R}$). In practical applications, it carries an associated measurement unit.

2.1 Core Properties of Scalars

1. Expresses magnitude only: No spatial orientation or coordinates are needed (e.g., 25 kilograms of rice).
2. Represented by a single real number: In algebraic notation, scalars are typically written using lower-case italic Roman letters: $a = 5, k = -2.5$.
3. Follows standard arithmetic operations: Scalar addition, subtraction, multiplication, and division follow conventional number field rules: $$5 + 3 = 8, \quad 4 \times 5 = 20, \quad 10 - 15 = -5$$
4. Dimensional homogeneity: In applied physics and engineering, only scalars sharing the same dimension and unit can be added or subtracted. Adding 5 kg to 3 kg yields 8 kg, but adding 5 kg to 10 seconds is physically nonsensical.

3 What is a Vector?

A Vector is a mathematical object uniquely characterized by both a Magnitude (length or norm) and a Direction (spatial orientation).

Classic examples from the physical world:

• Displacement: "10 meters Eastward".
• Velocity: "20 kilometers per hour heading North".
• Force: "15 Newtons directed downwards (gravitational pull)".

In mathematical notation, vectors are predominantly expressed in two standard ways:

3.1 Fundamental Vector Properties

1. Magnitude (Norm): The non-negative real scalar representing its length: $|\mathbf{v}| \ge 0$.
2. Direction: Orientation relative to chosen coordinate reference axes (frequently defined by angle $\theta$).
3. Dimensionality: A vector can exist in one dimension (1D), two dimensions (2D), three dimensions (3D), or arbitrary $n$-dimensional spaces ($n$D).
4. Components: Along each axis, a component can be positive, negative, or zero: $$\mathbf{v} = (3, -2)$$ Here, $+3$ denotes 3 units rightwards along the x-axis, and $-2$ denotes 2 units downwards along the y-axis.

4 Fundamental Comparison: Scalar vs. Vector

Scalars and vectors govern two distinct algebraic spaces. Their structural differences are summarized below:

5 Real-World Pairs: Scalar vs. Vector

Many physical quantities share identical units of measurement, leading to frequent confusion between their scalar and vector variants:

5.1 Distance vs. Displacement: A Concrete Walkthrough

Suppose an explorer leaves home and walks 10 meters Eastward. They then proceed another 5 meters Eastward.

• Total Distance: $10 + 5 = 15$ meters.
• Displacement: $15$ meters Eastward.

Now, suppose they turn around and walk 5 meters Westward back towards home.

• Total Distance: $10 + 5 + 5 = 20$ meters (distance strictly accumulates with every step).
• Displacement: The direct vector from start to final position = $15 - 5 = 10$ meters Eastward!

6 Vector Representation in Coordinate Systems

The most rigorous and computer-friendly method to represent vectors is through an orthogonal Cartesian Coordinate System.

In a 2D Cartesian plane, a vector $\mathbf{v}$ is decomposed into two orthogonal components: $$\mathbf{v} = (x, y)$$ where $x$ represents horizontal displacement and $y$ represents vertical displacement.

Column Vectors vs. Row Vectors

In linear algebra and matrix computations, vectors can be structured in two orientations:

7 Vector Dimension (Dimensionality)

The Dimension of a vector refers to the total number of independent numerical components required to define it.

8 Vector Magnitude (Euclidean Norm / $L_2$ Norm)

The Magnitude (or length) of a vector measures its geometric distance from the origin to its terminal tip.

For a 2D vector $\mathbf{v} = (x, y)$, since the $x$ and $y$ coordinate axes are orthogonal, we apply the Pythagorean Theorem directly:

Worked Example: If $\mathbf{v} = (3, 4)$, what is its magnitude? $$|\mathbf{v}| = \sqrt{3^2 + 4^2} = \sqrt{9 + 16} = \sqrt{25} = 5$$ The vector has a length of exactly 5 units.

8.1 3D and Higher-Dimensional Norm ($L_2$ Norm)

For a 3D vector $\mathbf{v} = (x, y, z)$, the Pythagorean theorem naturally extends: $$|\mathbf{v}| = \sqrt{x^2 + y^2 + z^2}$$

Example: For $\mathbf{v} = (2, 3, 6)$: $$|\mathbf{v}| = \sqrt{2^2 + 3^2 + 6^2} = \sqrt{4 + 9 + 36} = \sqrt{49} = 7$$

In machine learning and linear algebra, this universal length calculation for any $n$-dimensional real vector is formally known as the Euclidean Norm or $L_2$ Norm (denoted $\|\mathbf{v}\|_2$):

9 The Zero Vector

The vector whose components are all identically zero is called the Zero Vector (symbol: $\mathbf{0}$):

Two unique mathematical attributes of the Zero Vector:

• Zero length: $|\mathbf{0}| = \sqrt{0^2 + 0^2} = 0$.
• Undefined Direction: Because its initial and terminal points coincide, no direction angle can be assigned.

In linear algebra, the zero vector acts as the additive identity: adding it to any vector leaves the original unchanged: $\mathbf{v} + \mathbf{0} = \mathbf{v}$.

10 Equal and Opposite Vectors

11 Position Vectors and Displacement Vectors

12 Multiplying a Vector by a Scalar (Scalar Multiplication)

Multiplying a vector by a scalar $c$ scales each of its components independently by $c$: $$c \cdot \mathbf{v} = c \cdot (v_1, v_2) = (c v_1, \, c v_2)$$

13 Vectors and Data Representation in Machine Learning

Computers cannot natively comprehend raw physical entities (people, images, audio, documents). Computers understand numbers. Therefore, multiple measurable attributes of an entity are organized into an ordered vector known as a Feature Vector.

13.1 Real Estate Pricing Feature Vector ($5$D)

In housing price regression, a residential property is represented as:

13.2 Word Embeddings in NLP & LLMs

In modern Natural Language Processing and models like ChatGPT, human vocabulary is mapped into high-dimensional real vectors termed Vector Embeddings: $$\mathbf{v}_{\text{king}} = (0.24, \, -0.68, \, 0.91, \, \dots) \in \mathbb{R}^{768}$$

Within this continuous vector space, semantically related words reside close to one another geometrically. For example, the angle and distance between "King" and "Queen" vectors are remarkably small.

14 Vector Applications in Computer Graphics & Robotics

15 Scalars & Vectors in Programming (Python & NumPy)

In Python, scalars are stored as primitive numerical variables, while vectors are managed via tuples, lists, or most performantly using NumPy 1D Arrays:

import numpy as np

# 1. Scalar quantities
speed = 60          # Single numerical scalar
learning_rate = 0.01

# 2. Vector in Python tuple
pos_tuple = (3, 4)
print(pos_tuple[0])  # x = 3

# 3. Vector as a NumPy 1D Array
v = np.array([3, 4])

# 4. Euclidean Norm (Magnitude)
magnitude = np.linalg.norm(v)
print(f"Vector Magnitude: {magnitude}")  # 5.0

# 5. Scalar Multiplication (all components scaled simultaneously)
scaled_v = 2 * v
print(f"Scaled Vector: {scaled_v}")    # [6, 8]

16 Scalars & Vectors in AI & Machine Learning

17 7 Common Pitfalls to Avoid

19 Key Takeaways