Using Scalr Hooks with Bridgecrew Yor

What is Yor?

As the ecosystem around Terraform continues to grow we’re going to continue to blog about tools that we find interesting in this space. Today, we’ll talk about Bridgecrew Yor, which can be used to automatically tag infrastructure across a number of IaC platforms.

Yor supports a number of different “Built-in Taggers”. The  “Git-based Tagger” is super helpful for organizations adopting GitOps:

“The git_* tags connect cloud resources to individual git commits and establish clear ownership between developers and resources which are routinely change.”

The “custom tagger”, which we’ll interact with today, allows you to build any tagging standard to meet your organizational.

The combination of the two essentially brings together a necessary GitOps paper trail plus your tagging standard to meet the general organizational needs.

Integrate with Scalr

Scalr has many native integrations out of the box, but we have added flexibility by providing custom hooks to integrate with any tool in the cloud ecosystem, which is exactly what we’ll do with Yor. To do this we are going to leverage a couple of Scalr features:

• Custom hooks - This allows you to inject any command or script during the following Terraform phases:

• Pre-init
• Pre-plan
• Post-plan
• Pre-apply
• Post-apply

• Workspace Tags - These are tags associated with a workspace in Scalr,  but in this scenario we are going to take that value and use the Yor custom taggers to inject that tag to the Terraform resources as well.

Prerequisites:

• An account on scalr.io . We will be using the new beta UI, which can be accessed by adding /app2 to your URL: https://<account>.scalr.io/app2/
• Access to custom hooks on the pro tier.
• A VCS provider connected to Scalr.
• Terraform code in the connected VCS repository. The sample code below can be used.

Overview:

1. Create Terraform code if you don’t already have it.
2. Create the code which will pull Yor into the workflow and execute it.
3. Create the workspace settings and add the custom hook.
4. Execute the run and check the tags.

Steps

Create the Terraform Code

First, let's create some Terraform code to launch a S3 bucket. NOTE: You can use your own Terraform code, it is not required to use this sample as Yor will automatically tag any resources that it supports.

Sample Terraform code (main.tf):

resource "aws_s3_bucket" "example" {
  bucket = var.bucket_name

  tags = {
    Name        = "My bucket"
    Environment = "Dynamic Tagging"
  }
}

resource "aws_s3_bucket_acl" "example" {
  bucket = aws_s3_bucket.example.id
  acl    = "private"
}

Variables for the Terraform (variables.tf):

variable "bucket_name" {
  type = string
}

Code to Install Yor

Now lets create a pre-plan.sh script that will execute in the custom hook. This downloads Yor to be used in the Scalr runtime environment. 

#!/usr/bin/env bash

pip3 install requests click --quiet

export YOR_SIMPLE_TAGS="$(python3 scripts.py get-tags -h $SCALR_HOSTNAME -t $SCALR_TOKEN -r $SCALR_RUN_ID)"
wget https://github.com/bridgecrewio/yor/releases/download/0.1.151/yor_0.1.151_linux_amd64.tar.gz -q
tar -xf yor_0.1.151_linux_amd64.tar.gz
./yor tag --tag-groups simple -d ./ --skip-tags git*

NOTE: Make sure that pre-plan.sh is added as an executable to Git.

Now, let’s create a python script (scripts.py) that the pre-plan.sh will call. This script will pull a number of different objects like VCS information, run details, and workspace tags, which Yor will then use to tag the instance with:

import requests
import click
import json
import sys


@click.group()
def cli():
    """
    Scripts helper.
    """


@cli.command()
@click.option(
    "-h",
    "--hostname",
    type=str,
    multiple=False,
    help="The Scalr hostname",
)
@click.option(
    "-t",
    "--token",
    type=str,
    multiple=False,
    help="The Scalr token",
)
@click.option(
    "-r",
    "--run_id",
    type=str,
    multiple=False,
    help="The Scalr Run identifier",
)
def get_tags(hostname: str, token: str, run_id: str):
    def _fetch(route):
        response = requests.get(
            f"https://{hostname}/api/iacp/v3/{route}",
            headers={"Authorization": f"Bearer {token}", "Prefer": "profile=preview"}
        )
        document = response.json()
        if response.status_code != 200:
            raise Exception(json.dumps(document, indent=2))
        return document

    run = _fetch(f"runs/{run_id}?include=workspace,vcs-revision")

    run_attributes = run["data"]["attributes"]
    tags = {"run_id": run_id, "source": run_attributes["source"], "created-at": run_attributes["created-at"]}

    for val in run["included"]:
        attributes = val["attributes"]
        if val["type"] == "workspaces":
            ws_id = val["id"]
            tags.update({
                "workspace_id": ws_id,
                "workspace_name": attributes["name"],
                "environment_id": val["relationships"]["environment"]["data"]["id"],
            })

            if not val["relationships"].get("tags"):
                continue

            workspace_tags = []
            for ws_tag in _fetch(f"workspaces/{ws_id}?include=tags")["included"]:
                workspace_tags.append(ws_tag["attributes"]["name"])
            tags["workspace_tags"] = ','.join(workspace_tags)
        elif val["type"] == "vcs-revisions":
            tags.update(val["attributes"])


    print(json.dumps(tags))
    sys.exit(0)


if __name__ == "__main__":
    try:
        cli()
    except Exception as err:
        print(str(err))
        sys.exit(1)

Create a Scalr Workspace

Next, log into Scalr and create a workspace, in this example we’ll set a VCS provider as the source and link to the repository with the code that was just created:

Drop down custom hooks and add pre-plan.sh to the before plan hook:

Add some sample workspace tags. This can only be done in the new beta UI( add  /app2 to your URL https://<account>.scalr.io/app2/) or through the Scalr Terraform provider.

Click create workspace.

If you’re using the main.tf and variables.tf from the example, it will require you to set some variables in Scalr:

Execute the Run

Now all you have to do is execute the run:

During the run, you’ll see that it first executes the pre-plan.sh

Then the run goes through the rest of the pipeline resulting in an apply and the resources being tagged property in AWS:

In just a few simple steps we were able to automate our tagging strategy for Terraform using Scalr custom hooks and plugging Yor into the pipeline. As you can see, Yor is a very simple, yet powerful, tool to get up and running. The Scalr custom hooks open up the possibilities for many integrations and having a fully customized workflow without having to build a DIY pipeline.