How to use the Bitbucket Terraform Provider

Most infrastructure as code pipelines have two core parts: an IaC platform like Terraform or OpenTofu, and a version control system (VCS) like Bitbucket. Terraform defines and provisions infrastructure through a declarative configuration language, and pairing it with Bitbucket adds a layer of control and collaboration. People usually think of pull request automation when they hear "Bitbucket and Terraform," and the second half of this post covers that. First, though, we'll look at using Terraform to manage your Bitbucket configuration itself, plus the integration Scalr has with Bitbucket.

Understanding the Bitbucket Terraform Provider

What is the Bitbucket Terraform Provider?

Terraform uses the concept of providers to interact with endpoint APIs to create resources in the provider. The Bitbucket Terraform provider serves as a bridge between your infrastructure code and Bitbucket Cloud or Bitbucket Datacenter. It allows you to define and manage Bitbucket resources – such as repositories and projects – directly within your Terraform modules and code. This integration ensures that your repository configurations are codified, version-controlled, and can be easily reproduced across different environments.

Picking the right provider matters here. While many tools like Okta and GitLab have officially supported providers, Bitbucket is community driven, so you want the provider that is maintained and used the most. Searching the standard Terraform registry didn't help me much, since it gives you almost nothing to compare on.

Instead, I used library.tf, a registry maintained by Scalr, which gave me the insights I needed to make the decision on which provider to use:

Screenshot of Providers from Library.tf

As seen above, there is only one provider that has been updated in the last year and has frequent commits, when clicking on the provider I can also see the number of contributors, open issues, and more to help me make my decision on using this Bitbucket provider.

Prerequisites

Before getting into the common use cases of using the Bitbucket Terraform provider, ensure that you have the following prerequisites in place:

1. Terraform Installed: Make sure you have Terraform or OpenTofu installed on your machine. You can download the latest versions from Terraform or OpenTofu websites.
2. Bitbucket Cloud Account: You'll need a Bitbucket account with the necessary permissions to manage Bitbucket resources. If you don't have a Bitbucket account, sign up for one on the Bitbucket website. Bitbucket server is also an option, but we'll focus on Bitbucket cloud in this example.
3. Bitbucket secret (optional): Bitbucket supports authentication through a username/password, OAuth client ID and secret, or token.
4. Scalr Account: If you want to use Scalr as the remote backend, sign up for a free account here. Feel free to use this as a development environment while you explore Scalr.

Using the Bitbucket Provider: Examples

The next few steps cover how to authenticate to the provider, then walk through some basic examples of using it.

Define Required Providers and Provider Configuration

Start by configuring the Bitbucket Terraform provider in your Terraform code. Open your Terraform configuration file (commonly named main.tf) and add the following block:

Replace "your-username" with your Bitbucket username and "your-password" with the password you use for Bitbucket. The latest version of the Bitbucket Terraform provider documentation can be found here.

Manage Repos

Now, let's create a Bitbucket repository using the Bitbucket Terraform resource. This can all be viewed in detail in the Terraform registry here. Add the following code to your configuration:

This Terraform code defines a resource "example_repo" that is set to be private.

Manage Groups

In this example, we'll manage the groups in Bitbucket and what permissions they have. In this case, we'll also use a data source to pull the workspace ID:

This Terraform code defines a resource "example_group" and sets the groups permissions to read.

Run Terraform Commands

Once you've defined your Bitbucket resources through the Bitbucket Terraform provider, navigate to the directory containing your Terraform configuration file and run the following commands:

If you are using OpenTofu, use the tofu commands instead:

Terraform will initialize the Bitbucket provider and apply the changes to your Bitbucket account. Upon a successful Terraform run, the state file will be created.

Best Practices and Advanced Usage

Variables and Dynamic Configurations

Utilize Terraform variables to make your configurations more dynamic. Instead of hardcoding values, use variables to create reusable and flexible scripts.

Remote State Management

Consider using remote state management to store your Terraform state files securely. Services like Scalr or AWS S3 can be configured as remote backends to store state files. Here is an example of connecting to Scalr:

Data Sources

To improve your Terraform code, we encourage you to review the option of using Bitbucket data sources in the code to be able to pull information from other resources or workspaces into the run.

Provider Summary

So far we've covered the basic steps to configure the Bitbucket Terraform provider, create repositories and groups, and apply a few best practices. From here, the provider supports plenty of other Bitbucket resources worth a look, such as branches, deploy keys, and projects. The library.tf documentation for the provider is a good place to find more detail and examples.

Managing Bitbucket through Terraform lets you treat your VCS configuration the same way you treat the rest of your infrastructure as code.

Scalr's Integration with Bitbucket

Now let's switch gears and look at Bitbucket as part of your overall Terraform pipeline, and how it connects to Scalr.

Bitbucket as a VCS Repo

When talking about Terraform and Bitbucket, the number one best practice is to make sure your Terraform files are stored in a VCS repository, like Bitbucket. By storing the Terraform configuration in a Bitbucket repository, you enable the following for your infrastructure teams:

• Tracking Changes: A VCS keeps track of changes made to Terraform source code files within a Bitbucket repo. Each change is associated with a commit, which includes details such as who made the Terraform code changes, when it was made, and a description of the modifications.
• Collaboration: VCS enables multiple developers to work on the same Bitbucket repository simultaneously. Each developer can make changes independently, and the VCS helps merge these changes, preventing conflicts and ensuring a coherent Terraform codebase.
• History and Rollback: VCS maintains a complete history of all changes made to the Terraform code. This allows developers to roll back to previous versions of the Terraform code in case of errors or to investigate when a specific change was introduced.
• Branching and Merging: VCS allows developers to create branches, which are independent lines of Terraform code development. This feature is valuable for working on new features, bug fixes, or experimental changes without affecting the main Terraform codebase. By opening a merge request, you have the ability to automatically trigger a "Terraform dry run" in tools like Scalr or Terraform Cloud, which will execute a Terraform plan and send the results back into Bitbucket. Based on the results coming back successfully, developers can then merge the code allowing changes from one branch to be incorporated into another.
• Code Review: VCS facilitates Terraform code reviews by providing a clear view of the changes made in a commit. Reviewers can easily assess the modifications, leave comments, and suggest improvements before changes are merged into the main branch.

Using Bitbucket with a TACO (Terraform Collaboration and Automation Platform)

If you are looking for a TACO, such as Scalr or Terraform Cloud, to help scale your Terraform operations, the integration with Bitbucket is a key feature. By integrating these tools with Bitbucket, you are not only able to pull in the Terraform code but also enable the following features:

• Terraform Private Module Registry: On top of storing your Terraform code in Bitbucket for the reasons stated above, another best practice is to implement a Terraform Private Module Registry to store your Terraform modules. When doing this, you store your Terraform modules in Bitbucket repos, integrate Scalr or Terraform Cloud with Bitbucket and then pull the Terraform modules in for use by your DevOps teams. By doing this, Bitbucket is still the source of truth for the code, but you can now curate a list of approved modules for your teams to use.
• GitOps Workflow: A GitOps workflow for Terraform is usually the goal for most DevOps teams. A GitOps workflow can use the Bitbucket branching and merging to automatically kick off Terraform dry runs as well as Terraform applies. The flow is typically as follows: A branch, let's call it dev, is created based on the main branch. Terraform code is updated in the dev branch. A merge request is opened up against the main branch from dev. Based on the merge request being opened, the TACO will automatically execute a Terraform dry run and send the results back to Bitbucket pipelines. Admins can optionally (considered a best practice though) set branch policies to only allow a merge request if all checks come back successfully from the TACO. Once the merge request is approved the Terraform apply will execute in the TACO.
• Open Policy Agent: Open Policy Agent (OPA) with Terraform enables declarative policy enforcement for the Terraform code, allowing admins to define and enforce fine-grained policies to govern the provisioning workflow. Typically, the OPA policies are stored in Bitbucket and then pulled into Scalr or Terraform Cloud to enforce them in the core workflow.

The TACO products have a full feature set dedicated to Terraform, including the ability to store Terraform state files. See more here.