How to use the Azure DevOps Terraform Provider

There are a couple of core components in any infrastructure as code pipelines: An IaC platform, such as Terraform or OpenTofu, and a version control system (VCS) such as Azure DevOps, the VCS provider managed by Microsoft Azure. Terraform, a widely adopted IaC tool, allows users to define and provision infrastructure using a declarative configuration language. Integrating Terraform with Azure DevOps adds an extra layer of control and collaboration. When talking about Azure DevOps with Terraform, many think about pull request automation, which we will touch on in the second half of this blog, but we’ll also focus on how to use Terraform to manage your Azure DevOps configuration as well as the integration Scalr has with Azure DevOps

Understanding the Azure DevOps Terraform Provider

What is the Azure DevOps Terraform Provider?

Terraform uses the concept of providers to interact with endpoint APIs to create resources in the provider. The Azure DevOps Terraform provider serves as a bridge between your infrastructure code and Azure DevOps Cloud. It allows you to define and manage Azure DevOps resources – such as repositories and projects – directly within your Terraform modules and code. This integration ensures that your repository configurations are codified, version-controlled, and can be easily reproduced across different environments.

An important aspect of managing Azure DevOps with Terraform is to select the appropriate Terraform provider to use. There are many forks of the official provider out there, but using library.tf will allow you to see which provider is the original and officially supported provider compared to those that have been forked from it, for example:

By clicking on the banner at the top we are able to see the supported provider which we know is the one we need to use based on the statistics on the right-hand side:

Prerequisites to use it

Before getting into the common use cases of using the Azure DevOps Terraform provider, ensure that you have the following prerequisites in place:

1. Terraform or OpenTofu Installed: Make sure you have Terraform or OpenTofu installed on your machine. You can download the latest versions from Terraform or OpenTofu websites.
2. Azure DevOps Account: You'll need an Azure DevOps account with the necessary permissions to manage Azure DevOps resources. If you don't have a Azure DevOps account, sign up for one on the Azure DevOps website. 
3. Azure DevOps Person Access Token: Azure DevOps supports authentication through a personal access token, OIDC token, or a client secret. 

Scalr Account: If you want to use Scalr as the remote backend, sign up for a free account here. Feel free to use this as a development environment while you explore Scalr. You can also authenticate to Azure DevOps through the native integration.

Using the Azure DevOps Provider: Examples

In the following steps, we’ll provide you with the methods to authenticate to the provider and then walk through some basic examples of using it.

Define Required Providers and Provider Configuration

Start by configuring the Azure DevOps Terraform provider in your Terraform code. Open your Terraform configuration file (commonly named main.tf) and add the following block:

terraform {
  required_providers {
    azuredevops = {
      source = "microsoft/azuredevops"
      version = "1.0.1"
    }
  }
}

provider "azuredevops" {
  org_service_url = https://dev.azure.com/<your-org>             
  personal_access_token = <your-access token>
}

Replace "your–org” with the your actual Azure DevOps organization. You may need to update the complete URL depending on your organization settings. Update "your-access-token” with the access token you use for Azure DevOps. The latest version of the Azure DevOps Terraform provider documentation can be found here. 

Depending on your use case, your organization may not allow for the user of personal access tokens and you can use OIDC or client certificates instead.

Manage Repos

Now, let's create a new project in Azure DevOps using the Azure provider. This can all be viewed in detail in the Terraform registry here . Add the following code to your configuration:

resource "azuredevops_project" "scalr-example" {
  name               = "Scalr Example Project"
  visibility         = "private"
  version_control    = "Git"
  description        = "Scalr blog example"
}

This Terraform code defines an Azure DevOps project resource anmes "scalr-example" that is set to be private and use the git VCS protocol.

Create a Repository

In this example we’ll create a git repository in the Azure DevOps project that was created in the previous step. In this case, we’ll also use a datasource to pull the project ID:

resource "azuredevops_git_repository" "scalr-example" {
  project_id = azuredevops_project.scalr-example.id
  name       = "Scalr Blog Repository"
  initialization {
    init_type = "Clean"
  }
}

This Terraform code defines a resource that creates a repo in Azure DevOps named "scalr-example". This is a good way of creating all of your git repositories that will eventually store your Terraform files.

Run Terraform Commands

Once you've defined your Azure DevOps resources through the Azure DevOps Terraform provider, navigate to the directory containing your Terraform files and run the following commands:

terraform init
terraform plan
terraform apply

If you are using OpenTofu, use the tofu commands instead:

tofu init
tofu plan
tofu apply

Terraform will initialize the Azure DevOps provider and apply the changes to your Azure DevOps account. Upon a successful Terraform run, the Terraform state file will be created.

Best Practices and Advanced Usage

Variables and Dynamic Configurations

Utilize Terraform variables to make your configurations more dynamic. Instead of hardcoding values, use variables to create reusable and flexible scripts.

variable "url" {
 description = "The URL you authenticate to Azure DevOps with"
}

variable "token" {
 description = "The password you authenticate to Azure DevOps with"
}

provider "Azure DevOps" {
 org_service_url    = var.url
 personal_access_token   = var.token
}

Remote State Management

Consider using remote state management to store your Terraform state files securely. Services like Scalr or AWS S3 can be configured as remote backends to store state files. Here is an example of connecting to Scalr:

terraform {
  backend "remote" {
    hostname = "<account-name>.scalr.io"
    organization = "<scalr-environment-name>"

    workspaces {
      name = "<workspace-name>"
    }
  }
}

Data Sources

To improve your Terraform code, we encourage you to review the option of using Azure DevOps data sources in the code to be able to pull information from other resources or workspaces into the run.

In the example below, we show how to get a Azure DevOps repository using it’s name:

# First we need to pull the Azure DevOps project ID:
data "azuredevops_project" "example" {
  name = "Scalr Example Project"
}

# Set the project ID and then reference the name
data "azuredevops_git_repository" "example-single-repo" {
  project_id = data.azuredevops_project.example.id
  name       = "Scalr Blog Repository"
}

Provider Summary

In this first section of the blog, we covered the basic steps to configure the Azure DevOps Terraform provider, create projects, and repositories, as well as introduces the best practices for advanced usage. As you explore further, consider exploring additional resources supported by the Azure DevOps provider, such as branches, groups, service endpoints, and more. The library.tf documentation for the Azure DevOps Terraform provider is a valuable resource for in-depth information and examples.

By integrating Azure DevOps into your Terraform workflows, you're not just managing infrastructure – you're managing your VCS provider with the efficiency and scalability that infrastructure as code brings.

Scalr’s Integration with Azure DevOps

Now we’re going to switch gears a little and talk about how Azure DevOps can be used with your overall Terraform pipeline and how it integrates with Scalr.

Azure DevOps as a VCS Repo

When talking about Terraform and Azure DevOps, the number one best practice is to make sure your Terraform files are stored in a VCS repository, like Azure DevOps. By storing the Terraform configuration files in a Azure DevOps repository, you enable the following for your infrastructure teams: 

• Code Review: VCS facilitates Terraform code reviews by providing a clear view of the changes made in a commit. Reviewers can easily assess the modifications, leave comments, and suggest improvements before changes are merged into the main branch.
  
• Tracking Changes: A VCS keeps track of changes made to Terraform source code files within a Azure DevOps repo. Each change is associated with a commit, which includes details such as who made the Terraform code changes when it was made, and a description of the modifications.

• Collaboration: VCS enables multiple developers to work on the same Azure DevOps repository simultaneously. Each developer can make changes independently, and the VCS helps merge these changes seamlessly, preventing conflicts and ensuring a coherent Terraform codebase.
  

• History and Rollback: VCS maintains a complete history of all changes made to the Terraform code. This allows developers to roll back to previous versions of the Terraform code in case of errors or to investigate when a specific change was introduced.
  

• Branching and Merging: VCS allows developers to create branches, which are independent lines of Terraform code development. This feature is valuable for working on new features, bug fixes, or experimental changes without affecting the main Terraform codebase. By opening a merge request, you have the ability to automatically trigger a “Terraform dry run” in tools like Scalr or Terraform Cloud, which will execute a Terraform plan and send the results back into Azure pipelines. Based on the results coming back successfully, developers can then merge the code allowing changes from one branch to be incorporated into another.

Using Azure DevOps with a TACO (Terraform Collaboration and Automation Platform)

If you are looking for a TACO, such as Scalr or Terraform Cloud, to help scale your Terraform operations, the integration with Azure DevOps is a key feature. By integrating these tools with Azure DevOps, you are not only able to pull in the Terraform code but also enable the following features:

• Terraform Private Module Registry: On top of storing your Terraform code in Azure DevOps for the reasons stated above, another best practice is to implement a Terraform Private Module Registry to store your Terraform modules. When doing this, you store your Terraform modules in Azure DevOps repos, integrate Scalr or Terraform Cloud with Azure DevOps and then pull the Terraform modules in for use by your DevOps teams. By doing this, Azure DevOps is still the source of truth for the code, but you can now curate a list of approved modules for your teams to use.‍
• GitOps Workflow: A GitOps workflow for Terraform is usually the goal for most DevOps teams. A GitOps workflow can leverage the Azure DevOps branching and merging to automatically kick off Terraform dry runs as well as Terraform applies. The flow is typically as follows:
  • A branch, let’s call it dev, is created based on the main branch.
  • Terraform code is updated in the dev branch.
  • A merge request is opened up against the main branch from dev.
  • Based on the merge request being opened, the TACO will automatically execute a Terraform dry run and send the results back to Azure DevOps pipelines.
  • Admins can optionally (considered a best practice though) set branch policies to only allow a merge request if all checks come back successfully from the TACO.
  • Once the merge request is approved the Terraform apply will execute in the TACO.‍
• Open Policy Agent: Open Policy Agent (OPA) with Terraform enables declarative policy enforcement for the Terraform code, allowing admins to define and enforce fine-grained policies to govern the provisioning workflow. Typically, the OPA policies are stored in Azure DevOps and then pulled into Scalr or Terraform Cloud to enforce them in the core workflow.

The TACO products have a full feature set dedicated to Terraform, including the ability to store Terraform state files. See more here.