Getting Started with Terraform Providers

Today we are going to look into Terraform Providers, which sit at the foundation of how Terraform works. They are a big part of why it's so easy to use. The topics being covered are:

• What are they
• How to use them
• Tips/Tricks

What are they

Think of a Terraform Provider as a plugin that adds extra functionality, much like an extension in Visual Studio Code adds functionality to the editor. These Providers let us talk to basically anything that has an API. That's a big part of why Terraform is so useful: you get one consistent language and style for talking to almost anything you can think of.

Providers come in three different flavours (or Tiers):

• Official: owned and maintained by HashiCorp (e.g. vault)
• Partner: owned and maintained by a technology company that is a HashiCorp partner (e.g. azuredevops)
• Community: published and maintained by individual contributors or groups within the Terraform ecosystem (e.g. scratch)

Here's the process of the provider, shown in the below diagram.

Communication flow

1. HCL: the Terraform code we write to provision resources on a platform or service
2. Provider: the plugin for Terraform that takes the HCL, transforms it into the format required by the destination API
3. SDK: the client library used by the Provider to communicate with the API in Golang
4. API: the destination API that we want to communicate with

This pattern makes it really easy for us as engineers to talk to an API using Terraform, and it also makes building Providers for new APIs pretty simple. HashiCorp gives you a few ways to develop Providers.

Documentation is another big part of the Provider plugin. The docs about a Provider, its resources, and its data sources live right alongside the Provider code, and webpages can be generated from them automatically. Keeping those two things side by side makes it really easy for other engineers to pick up the Provider and start using it.

Providers usually come from a registry, either the one that ships with your TACO (most Terraform Cloud alternatives include a private registry) or the public HashiCorp registry, but you can also source them from the local filesystem. That's great when you're developing providers locally, or when you have internal providers that can't be hosted externally for security reasons.

How to use them

Now that we know what a Provider is and what it does, let's look at how to actually use one in our own Terraform code. Some basics are the same for every provider, but the attributes you need vary from provider to provider, which is why it's so important to check the provider's documentation. Let's start with the basics, then walk through some examples.

provider "some_provider" {}

Any provider declaration calls the provider block with the name of the provider as the ID, though the block isn't actually always required. If a provider needs no configuration (e.g. scratch, random) then you don't need a provider block at all. There are also cases where the provider has sensible defaults, or can pull its configuration from environment variables, so you don't have to declare a block. The google provider is a good example: if you define nothing, it looks for the configuration in environment variables or the gcloud cli configuration file(s).

It's good practice to keep a dedicated providers.tf file, since that makes it really easy to see every provider you're using straight from the code.

Here's an example with the azurerm provider for Microsoft Azure. For the most part this provider will try to pull its configuration from environment variables, but it MUST always have the block defined because of the features block it requires. Let's look at two ways of declaring this provider: one with the minimum requirements, and one with everything you need to get going when you're not sourcing from the environment.

This first example shows the simplest way to declare this provider. It just needs the features {} block to exist. Personally I think this requirement is ridiculous since it doesn't add any value and should be defaulted, but for now it's required.

provider "azurerm" {
	features {}
}

The next example shows what you need to pass into the provider so it can authenticate and access Azure resources. Here we need a Service Principal client_id and a client_secret, and along with those we also need to know which tenant and subscription to operate within.

provider "azurerm" {
	features {}
 
	client_id       = "00000000-0000-0000-0000-000000000000"
	client_secret   = ""
	tenant_id       = "10000000-0000-0000-0000-000000000000"
	subscription_id = "20000000-0000-0000-0000-000000000000"
}

Next up, let's look at the exact same scenario but for AWS. Thankfully AWS doesn't have something silly like the required features {} block 😃, we just pass an empty provider block and everything else gets sourced from the environment.

provider "aws" {}

The environment variables you need are AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_REGION. With those defined it can authenticate and talk to AWS. Next, let's look at the provider block with those properties defined in Terraform.

provider "aws" {
	region     = "ap-southeast-2"
	access_key = ""
	secret_key = ""
}

In both examples you can see that configuring providers isn't too cumbersome. For the most part it's about how to authenticate to a given platform or service. Another important part of using a provider is making sure it's up to date and sourced from the right location. Here's what you'd define in the terraform.tf file to constrain the provider.

terraform {
	required_providers {
		azurerm = {
			source  = "hashicorp/azurerm"
			version = "~> 3.0"
		}
 
		aws = {
			source  = "registry.trust.me/cloud/aws"
			version = "=5.31.0"
		}
	}
}

Here we're using the azurerm provider from the official HashiCorp registry and making sure we have the latest 3.x.x version. For AWS we're sourcing from a third-party trusted registry and asking for a specific version of the provider.

Tips/Tricks

The biggest and most important tip with providers is that you can have aliased versions of a provider. That means you can have a different version of the provider, different credentials, different... anything! This gets super powerful when you're managing infrastructure that spans multiple accounts and so on. Here's an example with two provider definitions for azurerm.

provider "azurerm" {
	features {}
}
 
provider "azurerm" {
	alias = "network"
 
	features {}
}

Now that we have our providers declared, we can also make sure they're constrained correctly, like this.

terraform {
	required_providers {
		azurerm = {
			source                = "hashicorp/azurerm"
			version               = "~> 3.0"
			configuration_aliases = [ azurerm.network ]
		}
	}
}

In the example below we're going to get a data source for a resource group using each provider. In a real-world Azure setup it's completely normal to have your networking live somewhere else under a different service account, and provider aliases let us reach both easily.

data "azurerm_resource_group" "app" {
	name = "rg-aue-prd-app"
}
 
data "azurerm_resource_group" "network" {
	provider = azurerm.network
 
	name = "rg-aue-prd-network"
}

Closing Out

So a provider is an easy way for Terraform, and by extension us as engineers, to talk to other platforms' and services' APIs using a language we all know. We went through how to actually use a provider, plus the most important tip of all: provider aliases, which let us have multiple connections to a single platform or service with different credentials or a different scope.