Kubernetes Terraform Provider

There are plenty of ways to deploy and manage what runs inside a Kubernetes cluster: a Helm chart, raw manifests, Kustomize, and so on. But in my experience, if your organization already uses Terraform (or OpenTofu), reaching for the Terraform Kubernetes provider to handle the full lifecycle of your cluster resources is usually the better call. It keeps the experience consistent for the people on your team today and for anyone who joins later. The HashiCorp configuration language (HCL) gives you one common language for describing and building any service or resource in your organization.

This post covers configuring the official Kubernetes provider to build out services across your environment no matter which cloud sits underneath, which is one of the main selling points of Kubernetes in the first place.

Provider Configuration

The Terraform registry is where you'll find the most up to date information about the Kubernetes Provider. A unique aspect of the Kubernetes provider is its use of versioned resources and data sources, these versions are aligned to the API version with Kubernetes itself and allows for strong backwards compatibility. The kubernetes secrets resources below are one example:

Additional information can be found out about the Kubernetes API and its versions here.

The provider gives you several ways to configure it, and they fall into two groups: explicit and implicit. Explicit means supplying configuration arguments directly in the provider block. Implicit means relying on environment variables. Either way, it is essential to think about sensitive information such as keys and tokens so you don't open up a security issue. I'd strongly encourage passing those sensitive details in via the CLI or environment variables. With those two groups in mind, here are some examples of authenticating the provider against a cluster:

• File configuration
• Credential configuration
• In-cluster configuration
• Exec plugins

File configuration

File configuration is a great way to configure the provider when working locally with Terraform as it simply takes a path to the Kubeconf which contains all relevant details, further to this you can constrain it to a given context or the default context will be used.

This would grab the relevant details from the Kubeconf to authenticate the provider with that particular cluster. Alternatively the environment variables KUBE_CONFIG_PATH and CONFIG_CONTEXT can be used.

Credentials configuration

This method of configuration allows for directly passing in the file contents for relevant authentication arguments, this approach can be very powerful when your Terraform orchestrator dynamically provides those files either for a single environment or different environments, allowing you to have the same configuration applied across multiple clusters. Here are those arguments:

These arguments are very self-explanatory, in that they do what they say. We are passing in the paths to the file builtin function which ingests the contents as a UTF-8 encoded string. This configuration can additionally be done by using data sources within a given cloud platform allowing for truly dynamic configuration of this provider.

In-cluster configuration

If you're running your Terraform commands within a Kubernetes cluster and it is that cluster you wish to execute on, the in-cluster configuration is perfect. This configuration is done via the KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT environment variables allowing Terraform to configure resources within the cluster.

Exec plugins

Cloud providers, when it comes to Kubernetes all give their consumers the ability to retrieve the Kubeconf from the managed clusters using CLI utilities. The Exec plugin type of configuration allows for exactly this scenario where you want to execute an external command to retrieve the correct details at runtime. The below example uses AWS' EKS service.

The contents of the exec block could easily be subbed out for the Azure variant:

In my opinion this is another unique feature of the Kubernetes Terraform provider, and a genuinely valuable one. It lets you reach external authentication mechanisms from inside the provider declaration itself.

Examples

The example below deploys a simple service into its own namespace using Terraform.

That config creates a service that distributes traffic across the three pods the deployment spins up, as the diagram below shows.

The advantage of provisioning Kubernetes services with OpenTofu/Terraform is that it's a common language teams understand regardless of what they happen to be deploying. You also get the features of OpenTofu/Terraform itself. The example above references the namespace instead of typing it out by hand, and it uses locals for the labels.

Closing Out

This post walked through the Kubernetes provider and how to set it up. We covered four ways to authenticate it: file configuration, credential configuration, in-cluster configuration, and exec plugins. Between them, engineers have enough flexibility to wire up Kubernetes however a given situation demands. We then finished with an example of using Terraform to deploy something into a cluster.