Why Use A Remote Operations Backend Over A Standard Backend

In a recent post, How to use Terraform remote backends, I covered what a backend is and the types Terraform users commonly reach for. This post looks at why you might pick a remote operations backend over a standard one. There are several remote operations backends on the market. Terraform Cloud and Scalr are the two I'll focus on here.

What Is a Standard Backend?

As covered in the previous post, standard backends are usually cloud-specific: an s3 bucket or Azure blob storage, for example. The diagram below shows the general components of a standard backend:

• State file(s)
• State versions
• State lock

What a standard backend gives you is a place to store state that's reachable from more than one machine. Every engineer can read the state and make changes to the Terraform code. Where versioning is enabled, previous versions of state are kept for that code; this is always a feature of the storage backend itself rather than a component of Terraform.

Standard backends are common and meet most companies' needs, but they push a lot of overhead onto engineers, who are now on the hook for orchestrating the Terraform code and running the other services around it. Sometimes that's exactly what you want. A team might run its own processes, such as testing, alongside each deployment. Using a remote operations backend doesn't stop you from doing that either.

What Do Remote Operations Backends Add on Top?

These used to be called enhanced backends and are now usually just remote. I prefer remote operations backends, because they let you store and version state and also run the Terraform code. For this post I'll stay off product-specific details and stick to what they have in common.

The below diagram outlines the components of a remote operations backend. In a similar way to the above standard backend, these components are as follows:

State

• State file(s)
• State versions
• State lock

Operations

• Runs
• Cost estimation
• Policy

In object-oriented terms, a remote operations backend is a child of the standard backend class. It has the full feature set of the standard one with extra goodies bolted on top. The three main benefits are:

• Runs (executions or orchestration)
• Cost estimation
• Policy enforcement and audit

Here is each one in detail.

Runs

These are the executions of the Terraform code, triggered either on checkin to the repository or by some other product-specific means. A run is also what wraps the other benefits of the remote operations backend, which we'll get to shortly.

Cost Estimation

Once the code has been planned, the remote operations backend also runs a cost estimation (where enabled). It tells the engineer the potential cost of either the entire environment or just the changes coming in with the latest commits. That's really handy, because engineers get to see the business cost of deploying resources to an organization's cloud before they do it. It's no longer an I didn't know situation.

Policy

Next comes the policy phase, where any policies the engineers have defined get run. This is how we restrict what gets deployed into cloud environments and how. It can go as far as controlling how a resource is configured or even named. That matters a lot in larger, regulated environments where security operations have to enforce specific controls. It also gives you evidence when an auditor wants proof that a given control is being followed.

You can write policy with Sentinel from HashiCorp or with OPA, which is part of the CNCF. Both give you very granular, precise control over the resources your Terraform code is creating. Scalr supports OPA; Sentinel is the HashiCorp-proprietary alternative used by Terraform Cloud.

However, this could also be something as mundane as ensuring the correct tags are put on resources!

So Which Backend Should You Choose?

We've now walked through the standard backend and the remote operations backend, what each one is and what it gives you. So what's the recommendation? Here it comes.

Whether you're just getting started with Terraform or already deep down the rabbit hole, I'd point you at the remote operations backends first. If your organization can afford the paid tiers, or you come in under the free tier that most products offer, it's a good place to start. You get all the features above, and you barely have to think about state management. The standard backend is perfectly fine too, and plenty of teams use it. It can feel like it gives you more flexibility. Even so, for most people I think the remote operations backend wins: you pick up a lot of features and cut down your operations overhead.

You can follow Brendan @BrendanLiamT on Twitter.