API Driven Workflows for Terraform and OpenTofu

Most people run Terraform or OpenTofu in Scalr through the CLI or a connected version control system (VCS). This post covers a less common path: triggering a run through the API. The difference is control. With a VCS-driven run, something in your repository sets the run off. With an API-driven run, you make the call yourself, so you decide exactly when it happens. Here are a few reasons you might want that.

Real-time or Event-Driven Infrastructure Provisioning

Sometimes a run has to fire the moment something happens, not on a schedule or a git push. An API-driven run can be triggered programmatically in response to an event, so your cloud infrastructure keeps up with what's actually going on. That matters most in fast-moving environments where waiting for someone to click a button isn't an option.

Integration with External Systems

When tight integration with external systems is required, using the API directly might be preferable. This is common in cases where Terraform needs to interact with custom scripts, external APIs, or other tools that aren't directly connected to the VCS provider. An API-driven approach allows for flexibility and customization in handling these integrations. For example, if an alert goes off in New Relic, you might have a specific workspace and configuration update that needs to be made in response to that alert. We also have a few customers integrate with Slack to kick off runs through Slack based on certain scenarios.

Custom Workflows and Complex Automation

Some automation goes past what a VCS-driven run can do on its own. When you need to chain several steps together, or wire a run into systems that have nothing to do with your repository, the API gives you room to do it. It is not uncommon to see the API approach with tools such as Github Actions, Harness, Bamboo, or Jenkins.

Here's how the API-driven workflow works.

Example

In this example, we are going to create a new workspace, upload the new configuration version (can be Terraform or OpenTofu), and then execute a new run.

Prerequisites

To start, you'll need to obtain values for the following objects in Scalr:

• API token
• URL for your Scalr account
• Workspace name
• Environment ID
• Terraform configuration files that are in a tar.gz file

Script

The below is a sample script that can be modified to your liking, but it will give you the basic steps to get started with. Make sure to update the token, base_url, env_id, ws_id, and upload_archive_path with values specific to your Scalr account. You can also update the is_dry_run flag based on the type of run you want to execute, a dry (speculative plan) run or a full Terraform apply.

import requests
 
token = ""
base_url = 'https://example.scalr.io'
headers = {
    'Prefer': 'profile=preview',
    'accept': 'application/vnd.api+json',
    'content-type': 'application/vnd.api+json',
    'Authorization': f'Bearer {token}'
}
env_id = ""
ws_id = ""
is_dry_run = True
upload_archive_path = ''
 
## Create CV
url = f'{base_url}/api/iacp/v3/configuration-versions'
data = {
    'data': {
        'attributes': {
            "auto-queue-runs": False,
        },
        'relationships': {
            'workspace': {
                'data': {
                    'type': 'workspaces',
                    'id': ws_id
                }
            }
        },
        'type': 'configuration-versions'
    }
}
 
response = requests.post(url, headers=headers, json=data)
 
cv_id = None
if response.status_code == 201:
    # Successful request
    result = response.json()
    # Process the response data
    print(result)
    cv_id = result['data']['id']
else:
    # Request failed
    raise Exception(f"Error: {response.status_code} - {response.text}")
 
upload_url = result['data']['links']['upload']
print(upload_url)
 
upload = requests.put(upload_url, headers={'Content-Type': 'application/octet-stream'}, data=open(upload_archive_path, 'rb'))
print(upload.status_code)
 
## create run
url = f'{base_url}/api/iacp/v3/runs'
data = {
    'data': {
        'attributes': {
            "is-dry": is_dry_run,
        },
        'relationships': {
            'configuration-version': {
                'data': {
                    'type': 'configuration-versions',
                    'id': cv_id
                }
            },
            'workspace': {
                'data': {
                    'type': 'workspaces',
                    'id': ws_id
                }
            }
        },
        'type': 'runs'
    }
}
 
response = requests.post(url, headers=headers, json=data)
if response.status_code == 201:
    # Successful request
    result = response.json()
    # Process the response data
    print(result)
else:
    # Request failed
    raise Exception(f"Error: {response.status_code} - {response.text}")

In this case we created a new workspace, but the same script can be used on an existing workspace. There are also a lot more options in terms of workspace settings that can control if the runs are executed immediately or wait until someone makes a separate request. All documentation on the Scalr API can be found here.

Summary

For a lot of teams, the best setup uses both. VCS keeps your infrastructure as code versioned and reviewable, and API-driven runs cover the dynamic, real-time changes that don't fit a git workflow. Which one you reach for depends on what a given workspace actually needs.