Why Use A Remote Operations Backend Over A Standard Backend

In a recent post, How to use Terraform remote backends, I covered what a backend is and the types Terraform users commonly reach for. This post looks at why you might pick a remote operations backend over a standard one. There are several remote operations backends on the market. Terraform Cloud and Scalr are the two I'll focus on here.

Standard Backends

As covered in the previous post, standard backends are usually cloud-specific: an s3 bucket or Azure blob storage, for example. The diagram below shows the general components of a standard backend:

• State file(s)
• State versions
• State lock

What a standard backend gives you is a place to store state that's reachable from more than one machine. Every engineer can read the state and make changes to the Terraform code. Where versioning is enabled, previous versions of state are kept for that code; this is always a feature of the storage backend itself rather than a component of Terraform.

Standard backends are common and meet most companies' needs, but they push a lot of overhead onto engineers, who are now on the hook for orchestrating the Terraform code and running the other services around it. Sometimes that's exactly what you want. A team might run its own processes, such as testing, alongside each deployment. Using a remote operations backend doesn't stop you from doing that either.

Remote Operations Backends

These used to be called enhanced backends and are now usually just remote. I prefer remote operations backends, because they let you store and version state and also run the Terraform code. For this post I'll stay off product-specific details and stick to what they have in common.

The below diagram outlines the components of a remote operations backend. In a similar way to the above standard backend, these components are as follows:

State

• State file(s)
• State versions
• State lock

Operations

• Runs
• Cost estimation
• Policy

In object-oriented terms, a remote operations backend is a child of the standard backend class. It has the full feature set of the standard one with extra goodies bolted on top. The three main benefits are:

• Runs (executions or orchestration)
• Cost estimation
• Policy enforcement and audit

Here is each one in detail.

Runs

These are the executions of the Terraform code triggered either on checkin to the repository or by some other product specific means. Runs are also what encapsulates the other benefits of the remote operations backend which we will discuss shortly.

Cost Estimation

When a plan of the code has been performed, the remote operations backend will also run a cost estimation (where enabled) that informs the engineer of the potential cost of either the entire environment or just the changes being put forward as part of the latest commits. This is an extremely useful feature for ensuring that engineers better understand the business cost of deploying resources to an organization's cloud environment. It no longer becomes an I didn't know situation.

Policy

The next phase is the policy phase; this is where any policies that the engineers have defined are run. The policy feature allows us as engineers to better restrict what and how resources are deployed into cloud environments; this may go so far as to restrict how a resource is configured or even named. Features like this are essential to larger, more regulated environments where security operations require the enforcement of specific controls. Furthermore, it also acts as a source of evidence in the case where an auditor might require proof that a given control is being adhered to.

Examples of the types of technology that can be used to write policy are Sentinel from HashiCorp and OPA which is part of the CNCF. Both of these technologies allow for incredibly granular and precise control of the resources that your Terraform code is creating. Scalr supports OPA; Sentinel is the HashiCorp-proprietary alternative used by Terraform Cloud.

However, this could also be something as mundane as ensuring the correct tags are put on resources!

Closing Out

We've now walked through the standard backend and the remote operations backend, what each one is and what it gives you. So what's the recommendation? Here it comes.

Whether you're just getting started with Terraform or already deep down the rabbit hole, I'd point you at the remote operations backends first. If your organization can afford the paid tiers, or you come in under the free tier that most products offer, it's a good place to start. You get all the features above, and you barely have to think about state management. The standard backend is perfectly fine too, and plenty of teams use it. It can feel like it gives you more flexibility. Even so, for most people I think the remote operations backend wins: you pick up a lot of features and cut down your operations overhead.

You can follow Brendan @BrendanLiamT on Twitter.