Terraform Expressions, Functions & Data Sources: A Comprehensive Guide

Terraform's power extends far beyond basic resource definitions. At its core lies a sophisticated expression language that transforms static configurations into dynamic, reusable, and maintainable infrastructure blueprints. This guide covers expressions, functions, data sources, local values, and meta-arguments—the essential building blocks for professional Infrastructure as Code.

Understanding Types and Type Constraints

Every value in Terraform has a type. Understanding these types is fundamental to writing correct configurations.

Primitive Types

• string: Textual data (e.g., "hello", "ami-0c55b31ad54347327")
• number: Numerical data (e.g., 100, 3.14)
• bool: Boolean values (true or false)

Collection Types

• list(...): Ordered sequence of elements (e.g., ["us-west-1a", "us-west-1c"])
• set(...): Unordered collection of unique elements
• map(...): Unordered key-value pairs (e.g., {Name = "MyInstance", Environment = "Dev"})

Structural Types

• object({...}): Structured data with named, typed attributes
• tuple([...]): Ordered sequence where each element has a specific type

Special Types

• null: Represents an absent or omitted value
• any: Accepts any type (use sparingly; bypasses type checking)

variable "subnet_ids" {
  type    = list(string)
  default = ["subnet-xxxxxxxx", "subnet-yyyyyyyy"]
}
 
variable "common_tags" {
  type    = map(string)
  default = {
    Terraform = "true"
    Project   = "Alpha"
  }
}
 
variable "config" {
  type = object({
    name    = string
    enabled = optional(bool, true)
  })
}

Working with Strings and String Interpolation

Strings support dynamic content through interpolation and template directives.

Basic String Interpolation

Embed expressions within strings using ${...}:

resource "aws_instance" "web" {
  tags = {
    Name = "Instance-${var.environment}"
  }
}

Template Directives

Use %{...} for conditional logic and loops:

locals {
  user_list = "%{ for user in var.users ~}${user}\n%{ endfor }"
 
  config = "%{ if var.enable_monitoring }MONITORING_ENABLED%{ else }MONITORING_DISABLED%{ endif }"
}

Heredoc Strings

For multi-line strings, use heredoc syntax:

locals {
  user_data_script = <<-EOT
    #!/bin/bash
    apt-get update
    apt-get install -y nginx
    systemctl start nginx
  EOT
}

Operators and Conditional Expressions

Comparison and Logical Operators

• Equality: == (equal), != (not equal) — type-strict, so 5 == "5" is false
• Comparison: >, >=, <, <= (for numbers)
• Logical: && (AND), || (OR), ! (NOT)
• Arithmetic: +, -, *, /, % (modulo)

Conditional Expressions

Select one of two values based on a boolean condition:

resource "aws_instance" "example" {
  instance_type = var.is_production ? "m5.large" : "t2.micro"
}
 
locals {
  backup_window = var.backup_window != null ? var.backup_window : "03:00-04:00"
}

For Expressions: Iteration and Transformation

Create new collections by iterating over and transforming existing ones.

List Output

output "instance_hostnames" {
  value = [for name in var.instance_names : "${name}.example.com"]
  # Result: ["web.example.com", "app.example.com", "db.example.com"]
}

Map Output

output "user_emails" {
  value = {for user in var.users : user => "${user}@example.com"}
  # Result: {"alice" = "[email protected]", "bob" = "[email protected]"}
}

Filtering with Conditions

output "even_numbers_doubled" {
  value = [for n in var.numbers : n * 2 if n % 2 == 0]
  # Result: [4, 8, 12]
}

Grouping Results

Use ... to group values into a list when keys might duplicate:

variable "servers" {
  type = list(object({ name = string, role = string }))
  default = [
    { name = "server1", role = "web" },
    { name = "server2", role = "app" },
    { name = "server3", role = "web" },
  ]
}
 
output "servers_by_role" {
  value = {for server in var.servers : server.role => server.name...}
  # Result: {"web" = ["server1", "server3"], "app" = ["server2"]}
}

Splat Expressions

A shorthand for extracting a list of attributes from a list of objects:

resource "aws_instance" "workers" {
  count         = 3
  ami           = var.ami_id
  instance_type = "t2.micro"
}
 
output "worker_ids" {
  value = aws_instance.workers[*].id
  # Equivalent to: [for inst in aws_instance.workers : inst.id]
}

The splat expression [*] is preferred over the legacy .* syntax. If the source is null, the result is an empty list; if it's a single object, it's treated as a single-element list.

Dynamic Blocks for Flexible Nested Configurations

Dynamic blocks construct repeatable nested configuration blocks without code duplication:

variable "ingress_rules" {
  type = list(object({
    port        = number
    protocol    = string
    cidr_blocks = list(string)
    description = optional(string)
  }))
}
 
resource "aws_security_group" "web_sg" {
  name = "web-server-sg"
 
  dynamic "ingress" {
    for_each = var.ingress_rules
    content {
      from_port   = ingress.value.port
      to_port     = ingress.value.port
      protocol    = ingress.value.protocol
      cidr_blocks = ingress.value.cidr_blocks
      description = ingress.value.description
    }
  }
}

Dynamic Block Components

• dynamic "<BLOCK_TYPE>": The nested block type (e.g., "ingress", "setting")
• for_each: Collection to iterate over
• iterator (optional): Custom name for the iteration variable (defaults to block type)
• labels (optional): Unique identifiers for each block instance
• content {}: Defines arguments for each generated block

Important: Dynamic blocks cannot generate meta-argument blocks like lifecycle. Use for_each on the resource itself for dynamic resource creation instead.

Comprehensive Function Reference

Terraform provides 80+ built-in functions organized by category. User-defined functions are not supported.

Function Basics

• Syntax: FUNCTION_NAME(ARG_1, ARG_2, ...)
• Argument Expansion: Use ... to expand lists into arguments: min(var.numbers...)
• Sensitive Handling: If a function argument is sensitive, the result is also marked sensitive
• Execution Timing: Most functions are "pure" (evaluated at plan time). Functions like timestamp() and uuid() produce unknown values at plan time, resolved during apply

Numeric Functions

For mathematical operations and calculations:

locals {
  instance_count = ceil(3.7)              # Result: 4
  floor_value    = floor(3.7)             # Result: 3
  max_disk       = max(60, 100, 80)       # Result: 100
  min_disk       = min(60, 100, 80)       # Result: 60
  power_value    = pow(2, 8)              # Result: 256
  int_value      = parseint("FF", 16)     # Result: 255
  absolute       = abs(-42)               # Result: 42
}

Available numeric functions: abs(), ceil(), floor(), max(), min(), pow(), parseint()

String Functions

Essential for name generation, formatting, and text manipulation:

locals {
  resource_name = lower(format("%s-%s", "MyApp", "Prod"))
  # Result: "myapp-prod"
 
  tags_string  = join(";", ["owner:team-a", "project:web"])
  # Result: "owner:team-a;project:web"
 
  project_code = replace("PROJ-WebApp", "/PROJ-/", "")
  # Result: "WebApp"
 
  trimmed      = trimspace("  hello  ")
  # Result: "hello"
 
  is_prod      = startswith(var.env, "prod")
}

Available string functions: format(), join(), split(), lower(), upper(), title(), substr(), replace(), trimspace(), startswith(), endswith()

Collection Functions

Working with lists, maps, and sets:

locals {
  zones       = ["us-west-2a", "us-west-2b", "us-west-2c"]
  zone_count  = length(zones)                    # Result: 3
  first_zone  = element(zones, 0)                # Result: "us-west-2a"
  all_zones   = concat(zones, ["us-west-2d"])
 
  default_config = {cpus = 2, memory = "4GB"}
  override_config = {memory = "8GB", network = "high-speed"}
  final_config = merge(default_config, override_config)
  # Result: {cpus = 2, memory = "8GB", network = "high-speed"}
 
  config_keys = keys(final_config)
  # Result: ["cpus", "memory", "network"] (sorted)
 
  unique_ports = toset([80, 443, 80, 8080])
  # Result: [80, 443, 8080] (no duplicates)
}

Available collection functions: length(), element(), concat(), flatten(), keys(), values(), lookup(), merge(), toset(), tolist(), tomap(), setproduct()

Encoding and Format Functions

Convert data between HCL and standard formats (JSON, YAML, Base64):

locals {
  iam_policy = {
    Version = "2012-10-17"
    Statement = [{
      Effect   = "Allow"
      Action   = "s3:ListBucket"
      Resource = "arn:aws:s3:::my-bucket"
    }]
  }
  policy_json = jsonencode(local.iam_policy)
 
  user_data = base64encode("#!/bin/bash\necho hello")
  decoded   = base64decode(local.user_data)
 
  compressed = base64gzip("large-content-here")
}

Available encoding functions: jsonencode(), jsondecode(), yamlencode(), yamldecode(), base64encode(), base64decode(), base64gzip(), urlencode()

Filesystem Functions

Read files from the local filesystem where Terraform executes:

locals {
  # Render a template with variables
  rendered_script = templatefile("${path.module}/user_data.tftpl", {
    server_role = "web-server"
    app_version = var.application_version
  })
 
  # Read SSH key
  ssh_key = trimspace(file(pathexpand("~/.ssh/id_rsa.pub")))
 
  # Check if file exists
  has_config = fileexists("${path.module}/custom.conf")
 
  # Read binary file as Base64
  binary_data = filebase64("${path.module}/binary.bin")
}

Available filesystem functions: file(), templatefile(), pathexpand(), fileexists(), filebase64()

Date and Time Functions

For timestamping, scheduling, and date formatting:

locals {
  now             = timestamp()
  plan_time       = plantimestamp()          # Terraform 1.5+
  formatted_date  = formatdate("YYYY-MM-DD", timestamp())
  expiry_date     = timeadd(timestamp(), "168h")  # 7 days
}

Available date functions: timestamp(), plantimestamp(), formatdate(), timeadd()

Hash and Crypto Functions

Generate hashes, UUIDs, and cryptographic operations:

locals {
  script_hash    = sha256(file("${path.module}/configure.sh"))
  md5_hash       = md5("my-string")
  stable_id      = uuidv5("dns", "my-service.example.com")
 
  password_hash  = bcrypt("my-password", 10)
 
  # For S3 object updates when content changes
  file_md5 = filemd5("${path.module}/script.sh")
}
 
resource "aws_s3_object" "script" {
  bucket = var.bucket_name
  key    = "scripts/configure.sh"
  source = "${path.module}/configure.sh"
  etag   = local.file_md5
}

Available hash functions: md5(), sha1(), sha256(), sha512(), filemd5(), filesha1(), filesha256(), filesha512(), uuid(), uuidv5(), bcrypt(), rsadecrypt()

IP Network Functions

Automate network configurations and subnet calculations:

locals {
  vpc_cidr          = "10.100.0.0/16"
 
  subnet_cidr       = cidrsubnet(local.vpc_cidr, 8, 0)
  # Result: "10.100.0.0/24"
 
  host_ip           = cidrhost(local.subnet_cidr, 10)
  # Result: "10.100.0.10"
 
  subnet_netmask    = cidrnetmask(local.subnet_cidr)
  # Result: "255.255.255.0"
 
  multi_subnets     = cidrsubnets(local.vpc_cidr, 4, 4, 4)
  # Result: ["10.100.0.0/20", "10.100.16.0/20", "10.100.32.0/20"]
}

Available IP functions: cidrhost(), cidrsubnet(), cidrnetmask(), cidrsubnets()

Type Conversion Functions

Explicitly convert values between Terraform types:

locals {
  bool_value      = tobool("true")           # "true" → true
  list_value      = tolist(toset([1, 2, 2]))  # Remove duplicates
  num_value       = tonumber("42")           # "42" → 42
  string_value    = tostring(123)            # 123 → "123"
 
  optional_sgs    = try(var.security_groups, [])
  is_valid_config = can(var.config.advanced.settings) && var.config.advanced.settings != null
}
 
variable "security_groups" {
  type    = list(string)
  default = null
}

Available conversion functions: tobool(), tolist(), tomap(), tonumber(), toset(), tostring(), try(), can()

Terraform Context Functions

Provide information about the execution environment:

locals {
  module_path     = path.module         # Current module's filesystem path
  root_path       = path.root           # Root module's path
  cwd_path        = path.cwd            # Original working directory
  workspace_name  = terraform.workspace # Current workspace name
 
  # Load workspace-specific config
  config_file = "${path.module}/configs/${terraform.workspace}.json"
 
  # Mark sensitive value
  secure_key = sensitive(var.api_key)
 
  # Check if value is sensitive (Terraform 1.8+)
  is_secret = issensitive(var.password)
}

Available context functions: path.module, path.root, path.cwd, terraform.workspace, sensitive(), nonsensitive(), issensitive()

Understanding Data Sources

Data sources fetch read-only information from external systems, cloud APIs, local files, or other Terraform states. They are declared with the data keyword and never modify infrastructure.

Key Characteristics of Data Sources

• Read-Only: Retrieve information without making changes
• Error Prevention: Validate external data existence during terraform plan
• Dynamic Configuration: Adapt to changing external data without hardcoding
• Modularity: Modules become self-sufficient by discovering environmental data
• Standard Interface: Consistent way to integrate with various systems

Data Source Syntax

data "<PROVIDER>_<TYPE>" "<LOCAL_NAME>" {
  # Configuration arguments (filters/identifiers)
  argument_name = expression
 
  # Outputs accessed as: data.<PROVIDER>_<TYPE>.<LOCAL_NAME>.<ATTRIBUTE>
}

Data Sources vs. Managed Resources

Managed resources (resource blocks) define infrastructure Terraform creates, reads, updates, and deletes (CRUD operations). Data sources (data blocks) provide read-only information used to configure those resources. An object should be managed by a resource OR referenced by a data source, not both in the same configuration.

The terraform_data resource is an exception—it stores arbitrary values in state without querying external systems.

Common Data Source Examples

Fetch Latest AMI

data "aws_ami" "latest_amazon_linux" {
  most_recent = true
  owners      = ["amazon"]
  filter {
    name   = "name"
    values = ["amzn2-ami-hvm-*-x86_64-gp2"]
  }
}
 
resource "aws_instance" "app" {
  ami           = data.aws_ami.latest_amazon_linux.id
  instance_type = "t3.micro"
}

Reference Existing VPC

data "aws_vpc" "selected" {
  id = var.target_vpc_id
}
 
resource "aws_subnet" "new_subnet" {
  vpc_id            = data.aws_vpc.selected.id
  cidr_block        = "10.0.1.0/24"
  availability_zone = var.az
}

Cross-State Data Sharing

data "terraform_remote_state" "network" {
  backend = "remote"
  config = {
    organization = "my-org"
    workspaces = {
      name = "prod-network"
    }
  }
}
 
resource "aws_instance" "app" {
  subnet_id = data.terraform_remote_state.network.outputs.subnet_id
}

Fetch Public IP

data "http" "my_public_ip" {
  url = "https://api.ipify.org?format=json"
}
 
locals {
  my_ip = jsondecode(data.http.my_public_ip.response_body).ip
}

Read Local File

data "local_file" "ssh_key" {
  filename = pathexpand("~/.ssh/id_ed25519.pub")
}
 
resource "aws_key_pair" "deployer" {
  key_name   = "deployer"
  public_key = data.local_file.ssh_key.content
}

Common Data Sources Reference

• aws_ami: Find AMI images by various filters
• aws_availability_zones: Get available zones in a region
• aws_vpc: Reference existing VPC
• aws_security_group: Find security groups
• terraform_remote_state: Access outputs from another Terraform state
• http: Fetch content from HTTP endpoints
• local_file: Read files from the local filesystem
• external: Call external programs and parse JSON output

Local Values (Locals)

Local values assign names to expressions, improving code readability and maintainability without exposing them as module inputs or outputs.

Benefits of Locals

• DRY Principle: Define a value once, reuse it everywhere
• Readability: Assign meaningful names to complex expressions
• Maintainability: Update logic in one place instead of multiple locations
• Module Scoping: Private to the module in which defined

Syntax and Usage

locals {
  project_prefix = "mycorp"
  environment    = "dev"
  region         = "us-east-1"
 
  # Locals referencing other locals
  common_name_prefix = "${local.project_prefix}-${local.environment}"
 
  # Complex calculations
  instance_count = var.enable_ha ? 3 : 1
 
  # Structured data
  common_tags = {
    Project     = local.project_prefix
    Environment = local.environment
    ManagedBy   = "Terraform"
    CostCenter  = var.cost_center
  }
}
 
resource "aws_instance" "web" {
  count         = local.instance_count
  instance_type = var.instance_type
 
  tags = merge(
    local.common_tags,
    {
      Name = "${local.common_name_prefix}-${count.index}"
    }
  )
}

Locals vs. Input Variables vs. Outputs

Breaking Down Complex Expressions

# Without locals—hard to read inline expression
resource "aws_security_group" "example" {
  tags = {
    Name = "sg-${join("-", [for part in split("-", var.environment): substr(part, 0, 1)])}-${var.application}"
  }
}
 
# With locals—clear and maintainable
locals {
  env_prefix = join("-", [for part in split("-", var.environment): substr(part, 0, 1)])
  sg_name    = "sg-${local.env_prefix}-${var.application}"
}
 
resource "aws_security_group" "example" {
  tags = {
    Name = local.sg_name
  }
}

Meta-Arguments: Powerful Resource Control

Meta-arguments are special arguments that modify resource behavior beyond their provider-specific configuration.

count for Resource Multiplication

Create multiple instances of a resource from a single block:

resource "aws_instance" "servers" {
  count         = 4
  ami           = var.ami_id
  instance_type = "t2.micro"
 
  tags = {
    Name = "Server-${count.index + 1}"
  }
}
 
resource "aws_subnet" "private" {
  count             = 3
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = element(var.availability_zones, count.index)
}
 
# Reference: aws_instance.servers[0].id, aws_instance.servers[1].id, etc.
output "instance_ids" {
  value = aws_instance.servers[*].id
}

Key Points:

• count.index: 0-based index of current instance
• count.value: Count value (useful with for_each transformations)
• Reference instances: resource_type.name[index]
• Use count for simple cases; use for_each when resources need meaningful identifiers

for_each for Key-Based Resource Creation

Create multiple instances identified by meaningful keys:

variable "subnets" {
  type = map(object({
    cidr = string
    az   = string
  }))
  default = {
    public_a = {cidr = "10.0.1.0/24", az = "us-east-1a"}
    public_b = {cidr = "10.0.2.0/24", az = "us-east-1b"}
    private_a = {cidr = "10.0.10.0/24", az = "us-east-1a"}
  }
}
 
resource "aws_subnet" "example" {
  for_each          = var.subnets
  vpc_id            = aws_vpc.main.id
  cidr_block        = each.value.cidr
  availability_zone = each.value.az
 
  tags = {
    Name = each.key
  }
}
 
# Reference: aws_subnet.example["public_a"], aws_subnet.example["private_a"], etc.
output "subnet_ids" {
  value = {for name, subnet in aws_subnet.example : name => subnet.id}
}

Key Points:

• each.key: Current iteration key
• each.value: Current iteration value
• Use for_each for resources needing stable, meaningful identifiers
• Cannot use count and for_each simultaneously on same resource

depends_on for Explicit Dependencies

Explicitly declare dependencies Terraform cannot infer:

resource "aws_instance" "app" {
  ami           = var.ami_id
  instance_type = var.instance_type
 
  depends_on = [
    aws_security_group.app,
    aws_iam_role.app_role
  ]
 
  iam_instance_profile = aws_iam_instance_profile.app.name
}
 
module "web_servers" {
  source = "./modules/web_servers"
 
  depends_on = [module.vpc, aws_security_group.web]
}

Best Practices:

• Use as last resort; prefer implicit dependencies through references
• Only for hidden dependencies (behavior dependencies, not data)
• Always document why the explicit dependency is necessary
• Common use cases: resource configuration ordering, bootstrapping, API limitations

lifecycle for Resource Management Strategy

Control creation, modification, and destruction behavior:

resource "aws_instance" "critical" {
  ami           = var.ami_id
  instance_type = "t3.large"
 
  lifecycle {
    create_before_destroy = true  # Create new before destroying old
    prevent_destroy       = true  # Block terraform destroy
    ignore_changes        = [tags["LastModified"]]  # Ignore specific changes
  }
}
 
resource "aws_autoscaling_group" "example" {
  launch_configuration = aws_launch_configuration.app.id
 
  lifecycle {
    create_before_destroy = true
    ignore_changes        = [load_balancers]
  }
}

Lifecycle Options:

• create_before_destroy: Create replacement before destroying old resource
• prevent_destroy: Block terraform destroy to prevent accidental deletion
• ignore_changes: Don't trigger updates when specified attributes change
• replace_triggered_by: Trigger replacement when other resources change

provider Meta-Argument for Provider Selection

Specify non-default provider for a resource:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}
 
provider "aws" "primary" {
  region = "us-east-1"
}
 
provider "aws" "backup" {
  region = "us-west-2"
}
 
resource "aws_s3_bucket" "backup" {
  bucket = "my-backup-bucket"
  provider = aws.backup
}

Best Practices for 2026

1. Modern Expression Usage

• Prefer for_each over count for meaningful resource identifiers
• Use splat expressions [*] for simple attribute extraction
• Leverage try() and can() for safe optional access
• Use optional() in variable types instead of lookup() and defaults

2. Function Application

• Extract complex expressions to locals for readability
• Use meaningful local variable names instead of inline functions
• Leverage templatefile() for script generation rather than inline templates
• Use jsondecode() and jsonencode() for structured data management

3. Data Source Practices

• Query dynamic information rather than hardcoding values
• Use data sources to validate external resource existence
• Combine data sources to create sophisticated filters
• Document data source dependencies that aren't obvious from code

4. Dynamic Blocks

• Use for repeatable nested blocks within single resource
• Prefer resource-level for_each for dynamic resource creation
• Keep dynamic block logic simple; extract complexity to locals
• Always test dynamic blocks thoroughly—they reduce immediate readability

5. Meta-Argument Usage

• Document depends_on with comments explaining the dependency reason
• Use lifecycle conservatively; prefer resource type defaults
• Combine for_each with lookup() for optional variable access
• Monitor resource count/iteration stability to prevent unnecessary replacements

6. Code Organization

• Group related locals and keep them near their usage
• Separate provider, variable, and local definitions into distinct sections
• Use modules to encapsulate related expressions and data sources
• Document complex expressions with inline comments

7. Performance Considerations

• Avoid expensive functions in repeatedly-evaluated expressions
• Use plantimestamp() instead of timestamp() when plan-time value suffices
• Cache expensive data source queries with locals
• Monitor plan time for expressions with many iterations

8. Security

• Mark sensitive values with sensitive() function
• Use try() to handle missing optional data safely
• Validate input with can() before accessing nested attributes
• Avoid exposing sensitive data through outputs

9. Type Safety

• Use specific type constraints instead of any type
• Leverage optional() for nullable object attributes
• Document type expectations in variable descriptions
• Use can() to validate expected structure before access

10. Testing and Validation

• Use validation blocks with custom conditions
• Apply preconditions to validate inputs before processing
• Add postconditions to verify resource state after provisioning
• Include test variables to exercise complex expressions

Conclusion

Mastering Terraform expressions, functions, data sources, local values, and meta-arguments transforms your IaC practice from simple infrastructure definitions to sophisticated, maintainable, and dynamic configurations. These building blocks enable:

• Code Reusability: Eliminate duplication through locals and modules
• Dynamic Adaptation: Query external systems through data sources
• Conditional Logic: Create flexible configurations with expressions
• Resource Multiplication: Scale configurations with count and for_each
• Advanced Patterns: Implement complex infrastructure requirements with dynamic blocks

Start by understanding the fundamentals—types, operators, and basic functions—then progressively leverage more advanced features as your infrastructure grows. Remember that clarity and maintainability matter as much as functionality. Well-written Terraform configurations that future team members can understand and modify confidently are the foundation of professional Infrastructure as Code practices.