Object Detection: YOLO, Bounding Boxes & Real-Time Computer Vision

1 Introduction: From Image Classification to Object Detection

In the previous lessons, we mastered Convolutional Neural Networks (CNNs) and Transfer Learning to classify images. An image classifier inspects an input tensor and outputs a single probability distribution predicting what object is present in the scene.

However, real-world visual perception demands far more than global image classification. Consider an autonomous vehicle approaching a busy intersection. The onboard camera captures a scene containing three vehicles, four pedestrians, two traffic lights, and a cyclist. An image classifier would merely output the label "street" or "car". It cannot tell the vehicle's planner:

• How many distinct objects exist in the scene?
• Where is each object located in 2D image coordinates?
• What is the spatial boundary (bounding box) around each independent obstacle?

2 Learning Objectives

3 Classification vs. Localization vs. Object Detection

In computer vision literature, the boundaries between classification, single-object localization, and general object detection are formally defined by the cardinality and structure of their output tensors.

4 Understanding Bounding Boxes & Coordinate Spaces

A bounding box represents the smallest axis-aligned rectangle (AABB) bounding an object. Different computer vision frameworks, annotation standards, and model architectures use different coordinate conventions. Mixing them up is the single most common cause of silent failures in vision pipelines.

5 Intersection over Union (IoU)

To evaluate how well a predicted bounding box $\mathbf{b}_{\text{pred}}$ matches a ground-truth bounding box $\mathbf{b}_{\text{gt}}$, we use the Jaccard similarity index, universally known in computer vision as Intersection over Union (IoU).

import torch

def calculate_iou(box1: torch.Tensor, box2: torch.Tensor) -> torch.Tensor:
    # Computes IoU between two tensors of shape [N, 4] and [M, 4] in (x1, y1, x2, y2) format.
    # Returns: [N, M] pairwise IoU matrix.
    # Compute intersection corners
    x1 = torch.max(box1[:, None, 0], box2[:, 0])
    y1 = torch.max(box1[:, None, 1], box2[:, 1])
    x2 = torch.min(box1[:, None, 2], box2[:, 2])
    y2 = torch.min(box1[:, None, 3], box2[:, 3])

    intersection = (x2 - x1).clamp(min=0) * (y2 - y1).clamp(min=0)

    area1 = (box1[:, 2] - box1[:, 0]) * (box1[:, 3] - box1[:, 1])
    area2 = (box2[:, 2] - box2[:, 0]) * (box2[:, 3] - box2[:, 1])
    union = area1[:, None] + area2 - intersection

    return intersection / (union + 1e-7)

6 How Does an Object Detector Work? (The 4-Pillar Pipeline)

Whether one-stage or two-stage, modern deep learning object detectors are modular architectures structured into four standardized stages:

7 One-Stage vs. Two-Stage Object Detectors

Object detection architectures historically diverged into two philosophical paradigms: two-stage proposal-driven systems and one-stage dense single-shot predictors.

8 Understanding YOLO: Evolution, Architecture & Paradigm

Introduced by Joseph Redmon et al. in 2015, YOLO ("You Only Look Once") revolutionized computer vision. Instead of repurposing classifiers to evaluate thousands of candidate crops, YOLO processes the entire image in a single neural forward pass.

9 Non-Maximum Suppression (NMS) & Duplicate Removal

Because dense object detectors evaluate thousands of overlapping grid cells simultaneously, multiple nearby cells will detect the same object, resulting in a cloud of redundant candidate bounding boxes. Non-Maximum Suppression (NMS) is the algorithmic filter used to retain only the single best detection per physical object.

10 Evaluating Object Detection Models: Precision, Recall & mAP

In object detection, classification accuracy is meaningless because bounding box accuracy is continuous. We evaluate detection systems using mean Average Precision (mAP) based on IoU matching criteria.

11 Hands-on Object Detection with Ultralytics YOLO in Python

Let us implement object detection using the official ultralytics library. We will load a pretrained model, run inference, parse prediction tensors, and process video streams.

from ultralytics import YOLO
import cv2

# 1. Load pretrained nano model (downloaded automatically if not cached)
model = YOLO("yolo11n.pt")  # or "yolov8n.pt"

# 2. Run inference on image with custom thresholds
results = model.predict(
    source="street_scene.jpg",
    conf=0.25,        # Confidence threshold
    iou=0.45,         # NMS IoU threshold
    imgsz=640,        # Square inference resolution
    device="cpu",     # Use "0" or "cuda:0" for GPU
    save=True         # Save annotated image to runs/detect/predict/
)

# 3. Parse bounding box coordinates and metadata
for r in results:
    boxes = r.boxes
    for box in boxes:
        # Bounding box in VOC absolute pixel format [x1, y1, x2, y2]
        x1, y1, x2, y2 = box.xyxy[0].tolist()
        
        # Confidence score and integer class ID
        conf = float(box.conf[0].item())
        cls_id = int(box.cls[0].item())
        cls_name = model.names[cls_id]

        print(f"[{cls_name.upper()}] Conf: {conf:.2f} | BBox: ({x1:.1f}, {y1:.1f}) -> ({x2:.1f}, {y2:.1f})")

# Process video or live camera (source=0) with memory-safe generator
results = model.predict(
    source=0,          # 0 for default webcam, or "traffic.mp4"
    conf=0.30,
    stream=True        # Essential: returns generator, avoids RAM exhaustion
)

for frame_result in results:
    # Render annotated frame with OpenCV
    annotated_frame = frame_result.plot()
    cv2.imshow("Real-Time YOLO Detection", annotated_frame)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cv2.destroyAllWindows()

12 Preparing a Custom Object Detection Dataset

Custom enterprise vision applications—such as PPE compliance, factory safety, or crop disease identification—require custom datasets. YOLO follows a strictly standardized directory layout and annotation syntax.

dataset/
├── images/
│   ├── train/ (img001.jpg, img002.jpg)
│   ├── val/   (img101.jpg)
│   └── test/  (img201.jpg)
├── labels/
│   ├── train/ (img001.txt, img002.txt)
│   ├── val/   (img101.txt)
│   └── test/  (img201.txt)
└── data.yaml

# Path relative to working directory or absolute
path: ./dataset
train: images/train
val: images/val
test: images/test

# Class ID mapping
names:
  0: helmet
  1: vest
  2: person

13 Training & Fine-Tuning YOLO on a Custom Dataset

Fine-tuning a pretrained model leverages existing low-level edge and texture filters while adapting upper detection heads to your domain-specific classes.

from ultralytics import YOLO

# 1. Initialize with pretrained weights (Transfer Learning)
model = YOLO("yolo11n.pt")

# 2. Train the detector
results = model.train(
    data="dataset/data.yaml",
    epochs=50,
    imgsz=640,
    batch=16,
    patience=10,               # Early stopping patience
    project="runs/ppe_detect",
    name="yolo11_ppe_v1",
    save=True,
    plots=True
)

# 3. Evaluate the best checkpoint on validation split
metrics = model.val(data="dataset/data.yaml", split="val")
print(f"Validation mAP@0.50:      {metrics.box.map50:.4f}")
print(f"Validation mAP@0.50:0.95: {metrics.box.map:.4f}")

# 4. Export to ONNX / TensorRT for high-speed edge deployment
model.export(format="onnx", dynamic=False, simplify=True)

14 Real-Time Computer Vision & Edge AI

In embedded systems and edge devices (e.g., NVIDIA Jetson Orin, Raspberry Pi 5 with AI Hat, edge industrial PCs), raw model accuracy must be balanced against latency budgets.

15 Object Detection vs. Object Tracking

Object detection is stateless: it detects objects in isolation frame by frame. Object Tracking (MOT - Multi-Object Tracking) maintains a consistent identity ($\text{Track ID}$) for each object across consecutive frames as it moves, turns, or experiences temporary occlusion.

# Seamless Multi-Object Tracking with persistent IDs
results = model.track(
    source="traffic_intersection.mp4",
    tracker="bytetrack.yaml", # or "botsort.yaml" for camera motion compensation
    conf=0.3,
    stream=True
)

for r in results:
    if r.boxes.id is not None:
        track_ids = r.boxes.id.int().tolist()
        print(f"Active Track IDs: {track_ids}")

16 Top 10 Object Detection Pitfalls & Anti-Patterns Matrix

17 Practical Assignment: Build an End-to-End Object Detection System

19 Key Takeaways