Understanding Terraform & OpenTofu Workspaces

As your Terraform projects grow in complexity, developers often face the challenge of maintaining multiple environments, such as development, staging, and production. This is where Terraform Workspaces come into play, offering a solution for managing different configurations and state files within a single Terraform project. In this blog, we will discuss what workspaces are, how to use them, and what the differences are when using a remote operations backend like Scalr or Terraform Cloud.

What are Terraform Workspaces

Terraform Workspaces allow you to create many instances of your infrastructure using the same Terraform code. Each workspace maintains its own state file, enabling you to manage different environments or configurations without duplicating your Terraform code. This feature is handy when you need to create and manage multiple instances of your infrastructure, such as for different stages of development or multiple clients.

The Basics of Workspaces

By default, Terraform operates in a workspace called "default." You're working within this default workspace when you run Terraform commands without specifying a workspace. However, you can create additional workspaces to manage different states of your infrastructure.

To create a new workspace, you can use the following command:

terraform workspace new <workspace_name>

This command creates a new workspace with the specified name and switches to it. You can list all available workspaces using:

terraform workspace list

To switch between workspaces, use:

terraform workspace select <workspace_name>

Benefits of Using Terraform Workspaces

1. Simplified Code Management: Workspaces allow using the same Terraform code across different environments. This reduces code duplication and makes maintaining consistency across your infrastructure easier.
2. Isolated State Files: Each workspace maintains its own state file, ensuring that changes in one environment don't affect others. This isolation helps prevent accidental modifications to production environments during development or testing.
3. Easy Environment Switching: With a simple command, you can switch between different environments, making it convenient to work on multiple configurations without changing your working directory or code.
4. Improved Collaboration: Team members can work on different environments simultaneously without interfering with each other's work, as each workspace has its own state.

Working with Terraform State in Workspaces

When you create a new workspace and execute a Terraform apply, Terraform generates a new state file specific to that workspace. The state file is crucial as it keeps track of the current state of your infrastructure and helps Terraform determine what changes need to be made to reach the desired state defined in your configuration.

In a local setup, Terraform stores these state files in a directory named terraform.tfstate.d within your working directory. Each workspace gets its own subdirectory named after the workspace, containing the state file for that particular workspace.

For example, if you have workspaces named "dev" and "prod," your directory structure might look like this:

├── main.tf

├── variables.tf

├── outputs.tf

├── terraform.tfstate.d

│   ├── dev

│   │   └── terraform.tfstate

│   └── prod

│       └── terraform.tfstate

└── .terraform

Remote State Management with Workspaces

While local state management works for small teams or individual projects, larger teams often benefit from using remote state storage. Terraform supports various backend types for storing state remotely, such as Scalr, Amazon S3, Azure Blob Storage, or HashiCorp's Terraform Cloud.

When using a remote backend with workspaces, the backend typically handles the separation of state files for different workspaces. For example, if you're using an S3 backend, you might configure it like this:

terraform {
 backend "s3" {
   bucket = "my-terraform-state"
   key    = "path/to/my/key"
   region = "us-west-2"
 }
}

In this setup, Terraform will automatically manage separate state files for each workspace within the specified S3 bucket. The actual path to the state file will include the workspace name, ensuring isolation between different environments.

Alternatively, in Scalr, you would configure it like this:

terraform {
  backend "remote" {
    hostname = "<account-name>.scalr.io"
    organization = "<scalr-environment-name>"


    workspaces {
      name = "<workspace-name>"
    }
  }
}

In this setup, a workspace is created in Scalr where the state is stored, run history is kept, and all items associated with that Terraform deployment are kept. In tools like Scalr and HCP Terraform Cloud, you aren’t just storing the state remotely; you are doing all operations remotely so teams can have a central place to deploy Terraform.

Using Workspace Names in Your Terraform Code

One of the powerful features of workspaces is the ability to use the workspace name within your Terraform code. This allows you to adjust your configuration dynamically based on the active workspace.

Terraform provides a special variable terraform.workspace that contains the name of the current workspace. You can use this variable to set values or make decisions in your Terraform code.

For example, you might use it to set different resource names or tags:

resource "aws_instance" "example" {
 ami           = "ami-0c55b159cbfafe1f0"
 instance_type = "t2.micro"

 tags = {
   Name = "instance-${terraform.workspace}"
   Environment = terraform.workspace
 }
}

In this example, the EC2 instance will be tagged with the name of the current workspace, making it easy to identify which environment it belongs to.

Workspace-Specific Variables

While the terraform.workspace variable is useful, you might want more fine-grained control over your configuration. You can achieve this by defining workspace-specific variables in your Terraform code.

For instance, you could create a map of variables for different environments:

variable "instance_type" {
 type = map(string)
 default = {
   default = "t2.micro"
   dev     = "t2.small"
   prod    = "t2.medium"
 }
}

resource "aws_instance" "example" {
 ami           = "ami-7654353"
 instance_type = var.instance_type[terraform.workspace]
}

In this setup, the instance type will be selected based on the current workspace, allowing you to easily manage different configurations for different environments.

Data Sources and Workspaces

Data sources in Terraform allow you to fetch information from existing resources or external services. When working with workspaces, you can use data sources to retrieve information specific to each environment.

For example, you might use a data source to fetch the appropriate VPC ID for each environment:

data "aws_vpc" "selected" {
 tags = {
   Environment = terraform.workspace
 }
}

resource "aws_instance" "example" {
 ami           = "ami-0c55b159cbfafe1f0"
 instance_type = "t2.micro"
 subnet_id     = data.aws_vpc.selected.id
}

This approach allows you to maintain environment-specific resources outside of your Terraform code while still referencing them appropriately in each workspace.

Best Practices for Using Terraform Workspaces

1. Use Consistent Naming Conventions: Adopt a clear and consistent naming convention for your workspaces. This makes it easier for team members to understand the purpose of each workspace.
2. Limit the Number of Workspaces: While workspaces are powerful, having too many can become confusing. Consider using separate Terraform configurations for significantly different environments or projects.
3. Use Remote State Storage: For team environments, always use remote state storage to ensure all team members are working with the most up-to-date state information.
4. Implement State Locking: Use a backend, such as Scalr or HCP Terraform Cloud, that supports state locking to prevent concurrent changes to the same code.
5. Document Workspace Usage: Clearly document how workspaces are used in your project, including any workspace-specific variables or configurations.
6. Use Scalr or Terraform Cloud for Advanced Features: For larger teams or more complex setups, consider using Scalr or  Terraform Cloud, which offer advanced workspace management features and collaboration tools.

Workspaces in Scalr or Terraform Cloud

The core concept of Terraform workspaces is similar between local usage and Scalr, but there are some important distinctions in how they're implemented and used:

Core Similarities:

1. Purpose: In both cases, workspaces are used to manage multiple instances of your infrastructure using the same Terraform configuration.
2. Isolation: Each workspace maintains its own state, allowing for separate environments or configurations.
3. Naming: Workspaces are identified by unique names in both local and Scalr setups.

Key Differences:

1. Implementation:
   • Local: Workspaces are essentially different state files in a local directory.
   • Scalr: Workspaces are more robust entities, encompassing state, variables, run history, and access controls.
2. Management:
   • Local: Managed via CLI commands (e.g. terraform workspace new, terraform workspace select).
   • Scalr: Managed through the web UI, API, or Terraform CLI with a remote backend configuration.
3. Scope:
   • Local: Workspaces are specific to a local working directory.
   • Scalr: Workspaces are environment-wide and can be accessed by team members with appropriate permissions.
4. Feature Set:
   • Local: Basic workspace functionality for state isolation.
   • Scalr: Enhanced features like variable sets, run triggers, PR automation, and team access controls.
5. State Storage:
   • Local: State files are stored on the local filesystem.
   • Scalr: State is stored remotely and managed by Scalr.

While the fundamental concept of isolating state and configurations is the same, Scalr expands on this idea, turning workspaces into more comprehensive units of management for your infrastructure. Scalr workspaces offer additional layers of functionality that aren't available in local workspaces, making them more suitable for team environments and complex infrastructures.

It's important to note that when migrating from local workspaces to Scalr, you're not just moving your state files – you're adopting a more feature-rich and collaborative approach to managing your infrastructure.

Conclusion

Terraform Workspaces provide a mechanism for managing multiple environments or configurations within a single set of Terraform code. By leveraging workspaces, you can simplify your infrastructure management, reduce code duplication, and improve collaboration within your team. Whether you're working on a small project or managing complex, multi-environment infrastructure, mastering Terraform Workspaces can significantly enhance your Infrastructure as Code practices.

As you continue to explore and implement Terraform Workspaces in your projects, remember that they are just one tool in the larger ecosystem of Terraform and infrastructure management. Always consider your specific needs and choose the approach that best fits your team's workflow and requirements.