15 min read Infrastructure Automation Part 3 (11 / 16)
Part 3 — CI/CD & IaC

Lesson 12.11 — Infrastructure as Code (IaC): What it is and why it's needed

Moving beyond manual console clicks: Automate, version-control, and provision servers, networks, security, and storage using declarative code

What we will master in this lesson:

In the previous lesson, we automated software builds, testing, and continuous deployment via CI/CD. However, if the underlying cloud infrastructure (VMs, VPCs, subnets, and firewalls) is still provisioned by clicking through browser dashboards, human error and environmental discrepancies are inevitable. Here is what we will achieve:

  • The crisis of manual "ClickOps" and the evolutionary rise of IaC
  • Declarative (Terraform) vs Imperative (Bash/CLI) paradigms
  • The IaC landscape: Terraform, CloudFormation, Pulumi & Ansible
  • Core Terraform Workflow: Write → Init → Plan → Apply → Destroy
  • Providers, Resources, Data Sources, Variables & Outputs
  • The State File, secrets hygiene, Remote State & DynamoDB locking
  • Detecting & reconciling Infrastructure Drift with plan -refresh-only
  • Real-world AWS hands-on: VPC, Security Group, and Nginx EC2 Instance
Real-World Scenario

1. The Crisis of Manual Cloud Management & The Need for IaC

Imagine you have built a modern Web Application and want to deploy it to a cloud provider like AWS, Google Cloud, or DigitalOcean. To get your system operational, you must log into the web console and configure several interconnected resources:

Compute & OS

Selecting VM sizes (EC2/Droplets), configuring base OS images, and binding SSH keypairs.

Network & VPC

Provisioning a dedicated Virtual Private Cloud (VPC), public subnets, and internet gateway routes.

Security & Storage

Setting Security Group ingress rules (ports 80/443), blocking private ports, and attaching NVMe block disks.

Creating a single server by clicking through forms in the cloud console is manageable at the beginning. But as your product scales, your engineering organization quickly demands three isolated environments:

  • Development: Where software engineers rapidly test experimental branch builds.
  • Staging: A production mirror used for QA verification and stakeholder demos.
  • Production: The live, mission-critical environment serving thousands of concurrent users.

Furthermore, to ensure low latency and high availability, you may need to replicate this identical setup across multiple global regions (e.g., US-East, Europe-Central, and Asia-Pacific).

The Catastrophic Failure Mode of "ClickOps":

Manually filling out dozens of web form fields leads inevitably to human error and Configuration Drift. For instance, suppose an engineer accidentally leaves database port 3306 exposed to the public internet (0.0.0.0/0) in Development, but locks it down in Production. The two environments now behave differently, leading to phantom bugs, broken staging parity, and massive security liabilities.

Can we instead define our entire datacenter, networking topology, security rules, and servers in plain text files and provision hundreds of cloud resources with a single deterministic command?

This is precisely what Infrastructure as Code (IaC) delivers.

Core Definition

2. What is Infrastructure as Code (IaC)?

Infrastructure as Code (IaC) is the practice of managing and provisioning servers, network topologies, storage volumes, databases, and firewall policies through machine-readable definition files, rather than physical hardware configuration or interactive web dashboard clicks.

The Software Engineering Analogy

Just as application code is tracked in Git, reviewed via Pull Requests, tested with automated suites, and deployed through pipelines, IaC treats your cloud infrastructure as first-class software code. Your entire cloud datacenter is documented and executable from a Git repository!

Consider defining a cloud virtual machine declaratively in a configuration file:

// Declarative Specification:
1. Provision an Ubuntu 22.04 LTS Virtual Machine.
2. Allocate 2 vCPUs and 4 GB RAM.
3. Attach a 50 GB encrypted NVMe SSD root storage volume.
4. Assign a public IPv4 address and allow ingress traffic on ports 80 and 443.

An IaC tool reads this blueprint, communicates directly with cloud provider REST APIs (AWS, GCP, Azure), and provisions the exact resources deterministically. By changing a single variable, you can stamp out identical Staging or Production environments in seconds.

Comparative Analysis

3. Manual Management (ClickOps) vs. IaC: 8-Dimension Comparison

Comparing manual cloud dashboard manipulation with code-driven IaC reveals staggering differences across reliability, velocity, and auditability:

Dimension Manual Infrastructure (ClickOps) Infrastructure as Code (IaC)
1. Provisioning Method Clicking buttons in browser consoles or ad-hoc terminal commands. Running declarative configuration files (HCL / YAML).
2. Repeatability Must be repeated manually from scratch for every new environment or region. Re-run the same script to generate 100 identical environments instantly.
3. Version Control Zero audit trail; impossible to know who modified which firewall rule or when. Tracked 100% via Git commits, branches, author tags, and pull requests.
4. CI/CD Integration Isolated and manual; cannot be integrated into automated release trains. Built directly into GitHub Actions or GitLab CI for hands-off deployment.
5. Environmental Parity Prone to human error; subtle configuration drift causes erratic bugs. 100% identical configuration guarantees predictable behavior across dev & prod.
6. Team Collaboration Engineers inadvertently overwrite each other's console modifications. Peer reviews, automated plan checks, and distributed State Locking.
7. Rollbacks & Recovery Must remember and manually undo dozens of console settings during outages. Revert the Git commit and run terraform apply to restore the prior state.
8. Documentation Maintained manually on wikis; becomes obsolete within days. The code is the living documentation—inspecting the repo reveals exact reality.
Important Rule: IaC does not automatically eliminate human bugs! If you commit a flawed configuration, IaC will faithfully provision flawed infrastructure. Validating syntax and reviewing terraform plan output before applying is non-negotiable.
Internal Mechanics

4. How IaC Works: Desired State vs. Current State

The core engine of modern IaC relies on comparing the Desired State (declared in your code) against the Current State (the live cloud environment):

// The Declarative IaC Lifecycle:
[Infrastructure Requirements]
           │
           ▼
[Configuration Code (.tf / YAML)] ──(Desired State)
           │
           ▼
      [Validation]                 ──(Syntax & schema verification)
           │
           ▼
    [Change Planning]              ──(Compares Current State vs Desired State)
           │
           ▼
     [Plan Review]                 ──(Engineer verifies diff before execution)
           │
           ▼
     [Apply Changes]               ──(Sends API calls)
           │
           ▼
  [Cloud Provider API]             ──(AWS / GCP / Azure REST endpoints)
           │
           ▼
[Live Cloud Infrastructure]        ──(VMs, VPCs, and Firewalls provisioned)

When you modify your code (e.g., upgrading instance RAM from 4 GB to 8 GB), the IaC engine does not destroy and recreate everything blindly. It computes the minimal delta required to bring the cloud into alignment with your desired state.

Architectural Paradigms

5. Declarative vs. Imperative IaC: Understanding the Difference

In the infrastructure ecosystem, there are two distinct design philosophies:

1. Declarative (Desired State)

What to build

You declare what the final infrastructure should look like. You do not write the procedural steps; the IaC tool determines how to achieve that state.

// Declarative (Terraform HCL):
resource "aws_instance" "web" {
instance_type = "t3.micro"
count = 3
}
Key Benefit: Strictly Idempotent. Executing it 10 times results in exactly 3 running servers. No duplicate machines are created.

2. Imperative (Step-by-Step)

How to build

You explicitly instruct the system which commands to run in sequential order (e.g., standard Bash or AWS CLI scripting).

# Imperative (Bash Script):
aws ec2 run-instances --count 3 ...
aws ec2 wait instance-running ...
aws ec2 create-security-group ...
Risk: Running this script a second time launches 3 additional servers, doubling your bill. Error handling and idempotency must be coded manually.
Ecosystem

6. The IaC Tooling Ecosystem: Terraform, CloudFormation, Pulumi & Ansible

Several industry-grade tools dominate the infrastructure automation landscape, each serving distinct requirements:

1. Terraform (HashiCorp) Industry Standard

The undisputed king of multi-cloud declarative IaC. Written in HashiCorp Configuration Language (HCL). Supports AWS, Azure, Google Cloud, Kubernetes, Cloudflare, and over 3,000 providers with a single consistent workflow.

2. AWS CloudFormation AWS Native

Amazon's native IaC service. Uses JSON or YAML templates. Zero state file management is required because AWS tracks it internally, but it cannot manage resources outside AWS.

3. Pulumi General Languages

Designed for developers who prefer standard programming languages (TypeScript, Python, Go, C#) over domain-specific HCL. Enables native loops, classes, and standard unit testing frameworks.

4. Ansible Configuration Mgmt

Specializes in post-boot server configuration. Uses agentless YAML playbooks to install packages, configure Nginx, and manage users. Paired with Terraform, it creates an end-to-end automation powerhouse.

Core Engine

7. How Terraform Works: The Core Workflow

In this curriculum, we will use Terraform to master IaC. Every Terraform deployment follows a clean 4-stage pipeline:

1. Write 2. Init 3. Plan 4. Apply
Step 1: Write (Authoring HCL) Declare desired infrastructure resources inside .tf files using HashiCorp Configuration Language.
Step 2: terraform init (Initialization) Downloads required provider plugins (e.g., AWS provider), configures the backend, and locks versions into .terraform.lock.hcl.
Step 3: terraform plan (Execution Blueprint) Generates a non-destructive dry-run diff comparing desired state against current state: + add, ~ change, and - destroy.
Step 4: terraform apply (Execution) Upon confirmation (yes), sends API calls to the cloud provider, provisions the physical resources, and writes the resulting state to terraform.tfstate.
Teardown: terraform destroy (Resource Cleanup) Tears down every managed resource in reverse dependency order to prevent cloud billing after lab experiments. Strictly restricted in production!
Building Blocks

8. The 7 Essential Components of Terraform

Mastering Terraform requires understanding these 7 foundational architectural components:

1. Configuration Files (.tf)

Standard modular project organization: main.tf (core resources), variables.tf (input declarations), outputs.tf (return values), and providers.tf (cloud authentication).

2. Provider (Plugin Interface)

Terraform delegates cloud-specific translation to providers. For instance, the AWS provider wraps official AWS SDK API endpoints:

provider "aws" {
  region = "ap-southeast-1" // Singapore
}

3. Resource (Infrastructure Building Block)

Declares physical or virtual objects to manage—VMs, subnets, route tables, or security groups:

resource "aws_instance" "web" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.micro"
}

4. Data Source (Read-Only Queries)

While resources create infrastructure, Data Sources fetch information about pre-existing cloud objects (such as the default VPC ID or latest official Amazon Linux AMI):

data "aws_vpc" "default" {
  default = true
}

5. Input Variables

Parameterize configurations to avoid hardcoding: var.instance_type allows dynamically adjusting sizes between Dev and Prod.

6. Output Values

Prints provisioned attributes to the terminal: EC2 public IPs, load balancer DNS names, or database connection strings.

7. State File (terraform.tfstate) — The Single Source of Truth

Terraform records metadata and maps declared resources to physical cloud identifiers (e.g., i-0a8b9c...) inside this critical JSON file. Without it, Terraform cannot determine what resources exist in your cloud!

Hands-on Lab

9. Building a Production-Grade Web Server on AWS with Terraform

Let's construct an end-to-end architecture: A custom VPC, Public Subnet, Internet Gateway, Route Table, Security Group (allowing HTTP port 80), and an EC2 Instance bootstrapping Nginx via user_data!

// Project Directory Layout:
terraform-project/
├── versions.tf        // Provider constraints
├── variables.tf       // Input variable declarations
├── main.tf            // VPC, Subnet, IGW, SG, EC2 resources
├── outputs.tf         // Public IP & URL outputs
├── terraform.tfvars   // Environment specific values
└── .gitignore         // Excludes state and secrets

Steps 1 & 2: Provider and Variable Configuration

// versions.tf
terraform {
required_version = ">= 1.5.0"
required_providers {
aws = { source = "hashicorp/aws", version = "~> 5.0" }
}
}
provider "aws" { region = var.aws_region }
// variables.tf
variable "aws_region"   { default = "ap-southeast-1" }
variable "instance_type" { default = "t3.micro" }
variable "project_name"  { default = "byteion-demo" }

Steps 3, 4 & 5: Declaring Infrastructure Resources

// main.tf — Network, Security, and Compute
// 1. Query latest official Amazon Linux 2023 AMI
data "aws_ami" "al2023" {
most_recent = true
owners      = ["amazon"]
filter { name = "name"; values = ["al2023-ami-2023.*-x86_64"] }
}

// 2. Dedicated VPC and Public Subnet
resource "aws_vpc" "main" {
cidr_block = "10.20.0.0/16"
tags = { Name = "${var.project_name}-vpc" }
}

resource "aws_subnet" "public" {
vpc_id                  = aws_vpc.main.id
cidr_block              = "10.20.1.0/24"
map_public_ip_on_launch = true
}

// 3. Internet Gateway and Public Route Table
resource "aws_internet_gateway" "main" { vpc_id = aws_vpc.main.id }
resource "aws_route_table" "public" {
vpc_id = aws_vpc.main.id
route { cidr_block = "0.0.0.0/0"; gateway_id = aws_internet_gateway.main.id }
}
resource "aws_route_table_association" "public" {
subnet_id = aws_subnet.public.id; route_table_id = aws_route_table.public.id
}

// 4. Security Group (Port 80 HTTP Ingress)
resource "aws_security_group" "web" {
name_prefix = "${var.project_name}-web-sg"
vpc_id      = aws_vpc.main.id
ingress { from_port = 80; to_port = 80; protocol = "tcp"; cidr_blocks = ["0.0.0.0/0"] }
egress  { from_port = 0; to_port = 0; protocol = "-1"; cidr_blocks = ["0.0.0.0/0"] }
}

// 5. EC2 Instance with Automated Nginx Bootstrap
resource "aws_instance" "web" {
ami                    = data.aws_ami.al2023.id
instance_type          = var.instance_type
subnet_id              = aws_subnet.public.id
vpc_security_group_ids  = [aws_security_group.web.id]
user_data = <<-EOF
#!/bin/bash
dnf install -y nginx
echo "<h1>Hello from Terraform IaC!</h1>" > /usr/share/nginx/html/index.html
systemctl enable --now nginx
EOF
}

Step 6: Declaring Output Values

// outputs.tf
output "public_ip"   { value = aws_instance.web.public_ip }
output "website_url" { value = "http://${aws_instance.web.public_ip}" }

Steps 7–11: Execution Commands & Validation

$ terraform init # Downloads AWS Provider plugins
$ terraform fmt && terraform validate # Verifies formatting and syntax
$ terraform plan # Inspects execution blueprint (+6 to add)
$ terraform apply # Confirm with 'yes'; live on AWS within 30 seconds!
Security Critical

10. Protecting the State File & Secrets Hygiene

The state file contains not only physical cloud IDs, but also plaintext database root passwords, private TLS keys, and initial token attributes.

Strict Security Mandate: Never commit terraform.tfstate to Git!

Exposing a state file in a Git repository leaks your complete cloud topology and passwords. Setting sensitive = true on an output variable only hides it from the CLI output—it remains fully visible in plaintext inside terraform.tfstate!

Always include a comprehensive .gitignore in your project root:

# .gitignore
.terraform/
*.tfstate
*.tfstate.*
*.tfplan
terraform.tfvars
Team Collaboration

11. Remote State & State Locking: Essential for Teams

Keeping the state file locally works for solo developers, but in a multi-engineer team, local states lead to catastrophic split-brain scenarios and state corruption.

Remote State (Amazon S3 / HCP)

The state file is stored in an encrypted central cloud bucket with versioning enabled, ensuring every engineer operates against the exact same reality.

State Locking (DynamoDB)

Prevents concurrent executions. When Engineer A runs apply, DynamoDB acquires a lock; if Engineer B runs a command simultaneously, it fails fast instead of corrupting state.

Drift Management

12. What is Infrastructure Drift & How to Fix It

Suppose an engineer logs into the AWS Web Console at midnight to debug an outage and opens Port 22 (SSH) manually on the Security Group, forgetting to remove it afterward.

The live cloud infrastructure now diverges from the code in Git. This condition is called Infrastructure Drift.

Detecting and Reconciling Drift:

During the next terraform plan, Terraform refreshes state against live cloud APIs, detects the unauthorized Port 22 rule, and proposes removing it to restore code compliance. Running terraform plan -refresh-only allows updating the state representation without making destructive modifications.

Modular Architecture

13. Terraform Modules: Reusable Code Architecture

Monolithic configuration files become unmanageable as systems scale. Terraform Modules encapsulate infrastructure subsets (e.g., network, web-server, and database) into parameterized packages:

// Instantiating the same module across environments:
module "dev_vpc" {
source     = "./modules/network"
cidr_block = "10.10.0.0/16"
}

module "prod_vpc" {
source     = "./modules/network"
cidr_block = "10.20.0.0/16"
}
Synergy

14. Provisioning vs. Configuration Management: The Terraform & Ansible Tandem

Terraform and Ansible are not competitors; they address distinct phases of the infrastructure lifecycle:

1. Provisioning (Terraform)

Constructing the physical building. Allocates VMs, subnets, firewalls, and managed cloud resources.

2. Configuration Management (Ansible)

Furnishing the interior. Connects into provisioned servers to install runtimes, configure Nginx, and manage systemd services.

The Industry Golden Pipeline:
[Requirements] → Terraform (brings up VMs) → Ansible (configures OS & software) → CI/CD (deploys app code) → Live Production!
Best Practices

15. The 10 Golden Rules of High-Performing IaC

1. Version Control Everything: Keep all infrastructure definitions committed to Git with descriptive messages.
2. Exclude State and Secrets: Leverage .gitignore to ensure .tfstate and credentials never leak.
3. Parameterize with Variables: Avoid hardcoding; configure environments using variables.tf.
4. Never Apply Without Plan: Carefully review destructive actions in terraform plan before confirming.
5. Mandatory Remote State & Locking: Utilize S3 + DynamoDB or HCP backends to prevent race conditions.
6. Principle of Least Privilege: Grant Terraform credentials only the specific IAM permissions required.
7. Modularize Repeatable Units: Abstract common architectures into reusable, tested modules.
8. Continuously Monitor Costs: Audit cloud resource pricing and instance sizing before executing applies.
9. Maintain Clean Formatting: Always run terraform fmt and validate prior to pushing.
10. Enforce GitOps Automation: Drive production modifications through automated CI/CD Pull Request gates.
GitOps Workflow

16. Integrating IaC with CI/CD: The GitOps Standard

Combining IaC with CI/CD establishes GitOps: managing infrastructure changes through Git Pull Requests.

1. Engineer submits an infrastructure modification via a Pull Request (PR).
2. GitHub Actions runs terraform fmt -check and terraform validate.
3. The pipeline generates terraform plan and comments the diff directly on the PR.
4. Staff architects review the blast radius and approve the merge.
5. Merging into main automatically triggers terraform apply in production!
Hands-on Lab

🎮 Interactive IaC & Terraform Workbench

Explore Terraform's lifecycle, compare declarative vs. imperative execution, test drift detection, and generate dynamic HCL blueprints in the interactive labs below.

Terraform Core Workflow Simulator

Execute stages sequentially to observe provider setup, dry-run plans, and resource provisioning.

READY
Progress: Step 1 / 4 (Initialization) 25%
bash — terraform-runner 0.0s 
$ terraform init
Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.41.0...
- Installed hashicorp/aws v5.41.0 (signed by HashiCorp)

Terraform has created a lock file .terraform.lock.hcl to record the provider selections it made above.

✔ Terraform has been successfully initialized!
You may now begin working with Terraform. Try running "terraform plan" to see any changes required for your infrastructure.
Summary & Next Steps

Key Takeaways: What We Have Mastered

  • The Power of IaC: Replaces error-prone manual web clicks with deterministic, version-controlled configuration files.
  • Declarative Philosophy: Focuses on describing the Desired State, ensuring idempotency and predictable execution.
  • Terraform Workflow: Author code → init → plan → apply → destroy.
  • Resources vs Data Sources: Resources create infrastructure, while Data Sources query pre-existing cloud objects.
  • State File Security: .tfstate holds plaintext secrets; it must never be committed to Git.
  • Remote State & Locking: Amazon S3 backends paired with DynamoDB state locks eliminate race conditions in teams.
  • Drift Detection: Unauthorized console modifications are identified and safely resolved via terraform plan.
  • Terraform + Ansible Synergy: Terraform provisions virtual hardware, while Ansible configures operating systems and applications.
Next Lesson Preview

Lesson 12.12 — Server Configuration, Secrets & Access Management

Now that we can spin up virtual machines in seconds with Terraform, we must secure them! In the next lesson, we will master disabling root logins, setting up dedicated sudo users, enforcing SSH key-pair authentication, configuring UFW firewall rules, and injecting credentials securely through Cloud Secrets Managers (such as AWS Secrets Manager and HashiCorp Vault).

Authoritative References & Documentation:
  • HashiCorp Terraform: Official Documentation & HCL Language Specification
  • HashiCorp Learn: Manage AWS Infrastructure with Terraform Tutorials
  • Terraform Best Practices: Structuring Projects, State Storage and Modules
  • AWS Architecture Center: Infrastructure as Code & CI/CD Pipelines
  • Martin Fowler: Infrastructure as Code Evolution & Declarative Infrastructure