Skip to content
Deadline to Die

IaC with Terraform

☁️ IaC with Terraform

Terraform allows you to define your data infrastructure (Buckets, Warehouses, Clusters) as code. This ensures consistency and reproducibility across environments (Dev, Staging, Prod).

🟢 Level 1: Foundations (The Workflow)

1. The HCL (HashiCorp Configuration Language)

Terraform uses a declarative syntax to describe resources.

resource "aws_s3_bucket" "data_lake" {
  bucket = "my-company-raw-data"
}

2. The Core Commands

  • init: Prepare the working directory.
  • plan: See what changes will be made before applying them.
  • apply: Execute the plan to create/update infrastructure.

🟡 Level 2: State & Modules

3. Terraform State

Terraform keeps track of the resources it creates in a State File. In production, this file must be stored remotely (e.g., in an S3 bucket with locking via DynamoDB).

4. Modules

Group related resources into reusable components. For example, a “Data Warehouse Module” that creates a Snowflake DB, Schemas, and Roles.

🔴 Level 3: Platform Engineering

5. CI/CD for Infrastructure

Automate your infrastructure changes using GitHub Actions or GitLab CI. Run terraform plan on every Pull Request and terraform apply on merge to main.

6. Provider-Specific Resources

Master the specific resources for your cloud:

  • AWS: Glue, Athena, Redshift.
  • GCP: BigQuery, Dataproc, Pub/Sub.
  • Azure: Synapse, Data Factory.

Never manually create resources in the Cloud Console for production. If it’s not in Terraform, it doesn’t exist.