Using Scalr Hooks with Bridgecrew Yor

What is Yor?

We keep an eye on tools in the Terraform ecosystem and write about the ones we find useful. This post is about Bridgecrew Yor, which automatically tags infrastructure across a number of IaC platforms.

Yor supports a number of different "Built-in Taggers". The "Git-based Tagger" is super helpful for organizations adopting GitOps:

"The git_ tags connect cloud resources to individual git commits and establish clear ownership between developers and resources which are routinely changed."*

The "custom tagger", which we'll interact with today, allows you to build any tagging standard to meet your organizational needs.

Using both together gives you the GitOps paper trail and your own tagging standard at the same time.

Integrate with Scalr

Scalr has many native integrations out of the box, but we have added flexibility by providing custom hooks to integrate with any tool in the cloud ecosystem, which is exactly what we'll do with Yor. To do this we are going to use a couple of Scalr features:

• Custom hooks - This allows you to inject any command or script during the following Terraform phases:

• Pre-init

• Pre-plan

• Post-plan

• Pre-apply

• Post-apply

• Workspace Tags - These are tags associated with a workspace in Scalr, but in this scenario we are going to take that value and use the Yor custom taggers to inject that tag to the Terraform resources as well.

Prerequisites:

• An account on scalr.io. We will be using the new beta UI, which can be accessed by adding /app2 to your URL: https://<account>.scalr.io/app2/
• Access to custom hooks, one of Scalr's core features — included on every plan.
• A VCS provider connected to Scalr.
• Terraform code in the connected VCS repository. The sample code below can be used.

Overview:

1. Create Terraform code if you don't already have it.
2. Create the code which will pull Yor into the workflow and execute it.
3. Create the workspace settings and add the custom hook.
4. Execute the run and check the tags.

Steps

Create the Terraform Code

First, let's create some Terraform code to launch an S3 bucket. NOTE: You can use your own Terraform code, it is not required to use this sample as Yor will automatically tag any resources that it supports.

Sample Terraform code (main.tf):

resource "aws_s3_bucket" "example" {
  bucket = var.bucket_name
 
  tags = {
    Name        = "My bucket"
    Environment = "Dynamic Tagging"
  }
}
 
resource "aws_s3_bucket_acl" "example" {
  bucket = aws_s3_bucket.example.id
  acl    = "private"
}

Variables for the Terraform (variables.tf):

variable "bucket_name" {
  type = string
}

Code to Install Yor

Now let's create a pre-plan.sh script that will execute in the custom hook. This downloads Yor to be used in the Scalr runtime environment.

#!/usr/bin/env bash
 
pip3 install requests click --quiet
 
export YOR_SIMPLE_TAGS="$(python3 scripts.py get-tags -h $SCALR_HOSTNAME -t $SCALR_TOKEN -r $SCALR_RUN_ID)"
wget https://github.com/bridgecrewio/yor/releases/download/0.1.151/yor_0.1.151_linux_amd64.tar.gz -q
tar -xf yor_0.1.151_linux_amd64.tar.gz
./yor tag --tag-groups simple -d ./ --skip-tags git*

NOTE: Make sure that pre-plan.sh is added as an executable to Git.

Now, let's create a python script (scripts.py) that the pre-plan.sh will call. This script will pull a number of different objects like VCS information, run details, and workspace tags, which Yor will then use to tag the instance with:

import requests
import click
import json
import sys
 
@click.group()
def cli():
    """
    Scripts helper.
    """
 
@cli.command()
@click.option(
    "-h",
    "--hostname",
    type=str,
    multiple=False,
    help="The Scalr hostname",
)
@click.option(
    "-t",
    "--token",
    type=str,
    multiple=False,
    help="The Scalr token",
)
@click.option(
    "-r",
    "--run_id",
    type=str,
    multiple=False,
    help="The Scalr Run identifier",
)
def get_tags(hostname: str, token: str, run_id: str):
    def _fetch(route):
        response = requests.get(
            f"https://{hostname}/api/iacp/v3/{route}",
            headers={"Authorization": f"Bearer {token}", "Prefer": "profile=preview"}
        )
        document = response.json()
        if response.status_code != 200:
            raise Exception(json.dumps(document, indent=2))
        return document
 
    run = _fetch(f"runs/{run_id}?include=workspace,vcs-revision")
 
    run_attributes = run["data"]["attributes"]
    tags = {"run_id": run_id, "source": run_attributes["source"], "created-at": run_attributes["created-at"]}
 
    for val in run["included"]:
        attributes = val["attributes"]
        if val["type"] == "workspaces":
            ws_id = val["id"]
            tags.update({
                "workspace_id": ws_id,
                "workspace_name": attributes["name"],
                "environment_id": val["relationships"]["environment"]["data"]["id"],
            })
 
            if not val["relationships"].get("tags"):
                continue
 
            workspace_tags = []
            for ws_tag in _fetch(f"workspaces/{ws_id}?include=tags")["included"]:
                workspace_tags.append(ws_tag["attributes"]["name"])
            tags["workspace_tags"] = ','.join(workspace_tags)
        elif val["type"] == "vcs-revisions":
            tags.update(val["attributes"])
 
    print(json.dumps(tags))
    sys.exit(0)
 
if __name__ == "__main__":
    try:
        cli()
    except Exception as err:
        print(str(err))
        sys.exit(1)

Create a Scalr Workspace

Next, log into Scalr and create a workspace, in this example we'll set a VCS provider as the source and link to the repository with the code that was just created:

Drop down custom hooks and add pre-plan.sh to the before plan hook:

Add some sample workspace tags. This can only be done in the new beta UI (add /app2 to your URL https://<account>.scalr.io/app2/) or through the Scalr Terraform provider.

Click create workspace.

If you're using the main.tf and variables.tf from the example, it will require you to set some variables in Scalr:

Execute the Run

Now all you have to do is execute the run:

During the run, you'll see that it first executes the pre-plan.sh

Then the run goes through the rest of the pipeline resulting in an apply and the resources being tagged properly in AWS:

That's the whole flow: a few steps to automate Terraform tagging by plugging Yor into the pipeline with Scalr custom hooks. Yor is quick to get running, and the same custom-hook approach works for plenty of other integrations. You get a customized workflow without building a DIY pipeline to host it.