Open Policy Agent (OPA) is a declarative policy language that can be used across your cloud ecosystem to ensure controlled deployments. It has increased in popularity with the Terraform community as a way to check Terraform plans and ensure DevOps teams are deploying according to organizational standards.
Parts one and two in this series provided an overview of developing and testing OPA policies and a detailed guide to writing OPA policies for Terraform and Scalr.
This article will explain the input data available to OPA from the Terraform Plan (tfplan) and the run time environment (tfrun).
NOTE: Some of the JSON code included in this article are just snippets and may not be a complete JSON object. Also in some places screenshots are used to show collapsed JSON structures.
The JSON file used in this article was generated by running terraform plan --out=FILENAME on the following configuration and then extracting the plan in JSON format using terraform show -json FILENAME
This configuration includes most common Terraform elements as follows.
Resource (with count =)
These elements all appear in the plan output and can therefore be used in OPA policies.The JSON created by terraform show has two main sections. tfplan is the representation of the plan itself, tfrun is details of the current run context created by Scalr.
tfplan is the JSON representation of the plan itself and consists of 6 sections, each of which is described in turn below.
What follows are detailed descriptions of the sections of tfplan that are most commonly used with OPA and a summary of the other sections.
The resource_changes section contains an array of all resources declared by the root and child modules. Each element of the array specifies the action(s) (create, update, delete, no-op) on each resource, and the before, after and after_unknown attribute values. The no-op action appears when before and after are identical and can be used to validate existing infrastructure against a new or changed policy.
Example JSON: Resource changes
change.actions can be [ “delete”, “create” ], i.e. 2 actions. This happens when Terraform is going to completely replace a resource as opposed to update in place.
Policies can check for specific attribute settings and check values against allowed lists from the after sections, or check whether specific attributes have been given a value using the after_unknown sections.
Example OPA: Check for an allowed value using change.after. This policy will pass because the required security group is in the vpc_security_group_ids array above.
r := tfplan.resource_changes[_]
vsg := r.change.after.vpc_security_group_ids[_]
not array_contains(vsg, required_sg)
reason := sprintf(
"%-40s :: security group %s must be included in list",
This prior_state section shows the state of existing resources prior to the plan being generated. It also shows the state of any data sources that were evaluated during terraform plan.
This section also shows the dependencies between the various resources and data sources.
Normally this section will only be used to check data source attributes and dependencies. Checking of resources should be done in the before/after sections of resource_changes
OPA can be used to enforce the use of data sources to set attribute values.
Example OPA: The customer needed to mandate the use of a specific, pre-created KMS key. The key id had to be obtained from a data source and on the resources the policy needed to ensure a data source was being used. This policy will pass because it is pulling the kms key from a data source.
tfrun.is_destroy == false
r := tfplan.configuration.root_module.resources[_]
r.type == "aws_ebs_volume"
r.mode == "managed"
kms_key := eval_expression(tfplan, r.expressions.kms_key_id)
not startswith(kms_key, "data.aws_kms_key.")
reason := sprintf("%-40s :: KMS Key not derived from data source (%s=%s) :: ",
Can be used to check the value of a variable but these values will have been evaluated in tfplan.resource_changes, so rarely used.
The output_changes section shows change of values for any outputs in the root module.
This planned_values section shows all of the outputs,resources, and their attributes for which the value is known at the time of the plan, as opposed to values that will be set during apply. This will be values that are specified through literals, variables and data sources and also values that have known defaults.
The tfrun section provides details of the run time environment in which the Terraform plan was created.
The table below provides a description of the typically useful elements of the tfrun data.
Workspace details, inlcuding tags
Enforce naming conventions
Name and ID
Details of the VCS repo the workspace is linked to, if any
Check allowed repos
Check author to prohibit unauthorised runs
Summary of the cost estimates created by Scalr
Prevent expensive plans
E.g. cli, vcs, manual
Allow or prohibit run types
True or false
Useful for disabling policy checks during a destroy
This concludes our series on OPA, if you have not read part one or two yet, please check them out. If you are interested in more examples, Scalr maintains an ever expanding library of OPA policy examples in our Github repository. Feel free to make a PR and contribute or create an issue if there is an example you would like to see.