Advanced Terraform Workflows with terraform_data

Most Terraform resources map to something real in a cloud account, like an aws_instance or an S3 bucket. But some don't. Terraform 1.4 added the terraform_data managed resource, which creates no infrastructure at all.

So why use it? terraform_data holds arbitrary data in your state, gives provisioners a place to run when no other resource fits, and can force other resources to be replaced when a value you care about changes. This post walks through how it works and where it earns a spot in a real configuration.

What is terraform_data?

terraform_data is a managed resource that lives in your Terraform state. It's there to do three things:

1. Store arbitrary data: Values you define are stored in the state and can be referenced elsewhere.
2. Trigger provisioners: It can act as a dedicated resource to run scripts when no other infrastructure resource is a logical fit.
3. Influence resource lifecycles: It can be used with the replace_triggered_by lifecycle argument to force the replacement of other resources based on changes to arbitrary values.

It's always available through the built-in terraform.io/builtin/terraform provider, so you don't have to install or configure an external provider.

Key Arguments and Attributes:

• input (Optional): Accepts any value (string, number, list, map). Changes to this value cause Terraform to plan an update for the terraform_data instance.
• triggers_replace (Optional): Also accepts any value. A change here forces the terraform_data resource to be replaced (destroyed and recreated), which is key for re-running provisioners.
• output (Attribute): Reflects the value of the input argument, making it accessible to other parts of your configuration.
• id (Attribute): A unique ID for the resource instance, typically a UUID.

The Evolution: From null_resource to terraform_data

Before terraform_data, people usually reached for the null_resource from the hashicorp/null provider. terraform_data is its built-in successor, and it improves on it in a few ways:

For new configurations on Terraform 1.4+, terraform_data is the recommended choice.

Core Use Cases with Examples

Here is what terraform_data looks like in practice.

1. Storing Arbitrary Data with Resource Lifecycle

You can centralize configuration data that needs to be part of the Terraform state:

resource "terraform_data" "configuration_params" {
  input = {
    region        = "us-east-1"
    instance_size = "m5.large"
  }
}
 
resource "aws_instance" "example" {
  ami           = "ami-0c55b31ad2c359908" # Example AMI
  instance_type = terraform_data.configuration_params.output.instance_size
  # ... other configurations
}
 
output "configured_region" {
  value = terraform_data.configuration_params.output.region
}

A change to terraform_data.configuration_params.input will update this resource, and potentially the aws_instance if it consumes the output.

2. Triggering Provisioners

When you need to run scripts (e.g., local-exec or remote-exec) that aren't tied to a specific piece of infrastructure, terraform_data is a good place to put them:

resource "aws_instance" "web" {
  # ... configuration for web server
}
 
resource "aws_db_instance" "database" {
  # ... configuration for database
}
 
resource "terraform_data" "bootstrap_application" {
  triggers_replace = [
    aws_instance.web.id,
    aws_db_instance.database.id,
  ]
 
  provisioner "local-exec" {
    command = "./scripts/deploy_app.sh ${aws_instance.web.public_ip} ${aws_db_instance.database.address}"
  }
}

Here, if the web or database instance ID changes (signifying replacement), the terraform_data.bootstrap_application resource is also replaced, re-running the deployment script. While powerful, remember that provisioners should generally be a last resort; prefer managing resources declaratively where possible.

Advanced Patterns

Integrating with replace_triggered_by

One pattern that works well is using terraform_data to trigger the replacement of another resource based on arbitrary conditions:

variable "app_version" {
  type    = string
  default = "1.0.0"
}
 
resource "terraform_data" "version_tracker" {
  input = var.app_version
}
 
resource "aws_ecs_service" "my_app_service" {
  name            = "my-app"
  cluster         = aws_ecs_cluster.my_cluster.id
  task_definition = aws_ecs_task_definition.my_app_task.arn
  desired_count   = 3
  # ... other configurations
 
  lifecycle {
    replace_triggered_by = [
      terraform_data.version_tracker
    ]
  }
}

If var.app_version changes, terraform_data.version_tracker is updated. The lifecycle block in aws_ecs_service.my_app_service detects this change and plans a replacement for the service, effectively rolling out the new version.

Dependencies and lifecycle rules like this get harder to track as a team grows. Platforms like Scalr can give you visibility and governance over these Terraform configurations, helping keep them in line with operational best practices and organizational policies through features like customizable OPA policies.

Using terraform_data with for_each and count

You can create multiple terraform_data instances using for_each or count. But be careful when you use a collection of terraform_data resources in replace_triggered_by for another resource collection. A change in one terraform_data instance might accidentally trigger the replacement of all instances in the dependent collection. Test this carefully before you rely on it.

With iterated resources and trigger mechanisms like these, you want to know the blast radius of a change before you apply it. Tools that offer environment management and detailed run previews, like those in Scalr, can give you a better picture before you apply changes that might reach further than you expect.

Best Practices and Considerations

• When to Use: Ideal for triggering provisioners without a natural host resource, storing lifecycle-bound data, or as an intermediary for replace_triggered_by.
• Sensitive Data: Data in input is stored in plain text in the Terraform state. Secure your state backend rigorously. Provisioners should fetch secrets from secure stores (e.g., HashiCorp Vault, AWS Secrets Manager) at runtime rather than receiving them as direct inputs.
• Idempotency: Ensure provisioner scripts are idempotent (running them multiple times yields the same result without errors).
• Unknown Values: If input depends on a yet-to-be-created resource, its output will be unknown during the plan phase. This can cause issues if used in count or for_each of other resources.
• Avoid Overuse: Don't let terraform_data become a crutch for overly complex imperative logic. Strive for declarative configurations.

As configurations grow with resources like terraform_data, you need a way to enforce security policies around things like sensitive data handling or provisioner usage. Integrating policy-as-code frameworks, such as Open Policy Agent (OPA) managed through platforms like Scalr, allows organizations to codify and automatically enforce these standards across all Terraform operations.

terraform_data vs. locals

It's worth keeping terraform_data and local values (locals) straight:

• locals: Compile-time conveniences for naming expressions. They don't have a lifecycle and aren't stored independently in the state. They cannot directly trigger provisioners or be used in replace_triggered_by.
• terraform_data: Actual resources with a lifecycle, persisted in the state. They can trigger provisioners and be used in replace_triggered_by.

Use locals for simplifying expressions and terraform_data when you need resource-like behavior for data or actions.

When to reach for terraform_data

terraform_data handles a few specific problems well. It holds data inside a resource lifecycle, hosts provisioners that have no natural home, and drives replacement through replace_triggered_by. None of those need an external provider anymore.

The catch is that the same flexibility makes it easy to reach for when a local would do. Watch the pitfalls with collections and sensitive data, keep the imperative logic to a minimum, and it stays a clean addition to your configuration. As your Terraform usage grows, an IaC management platform can help you apply these patterns consistently across teams.