The Terraform & OpenTofu Terralith

A Terraform or OpenTofu Terralith refers to a Terraform configuration where all or most of the infrastructure is defined in a single, large root module. In this approach, you'll find a huge collection of Terraform resources, input variables, and outputs, all residing in a handful of .tf files within the same directory.

We doubt anyone starts out with the goal of having one massive module that does it all, but rather, it starts as a simple module that continually grows with minor additions. It's straightforward to get started, as everything is in one place. You have a main.tf file that defines your resources, a variables.tf file for your input variables, and perhaps an outputs.tf for any outputs you need. However, as your infrastructure and Terraform adoption grows, this approach can quickly become unwieldy and difficult to manage.

Why You Should Avoid a Terralith

In the section below, we’ll discuss why you should avoid a Terralith and what the best practices are instead:

First off, the blast radius of a Terralith can be massive. You make one minor change to a simple resource, but you now have to think of all of the cascading effects it might have on all other resources within the module.

Instead, you should break it down into multiple purposefully built modules.  Terraform modules allow you to create reusable components that can be pieced together to avoid a massive blast radius issue. Without modules, you miss out on the benefits of code reuse and abstraction. Without the ability to create reusable modules, you're more likely to end up with duplicate code across your configuration. This not only violates DRY (Don't Repeat Yourself) principles but also makes updates and maintenance more time-consuming and error-prone.

That being said, root modules are not bad and should be used. Your root modules should be relatively simple, primarily calling other modules and passing variables. This keeps your main configuration clean and easy to understand at a high level.

Next, think about the speed of your development cycles. With a Terralith, your Terraform plan and apply speeds will be very long as Terraform will need to check all existing resources against the Terralith. Even one small change requires planning and testing the entire infrastructure. This slows down your development velocity, which breaks the whole idea behind DevOps. Instead, break the deployments down into smaller reusable modules and workspaces that execute quicker and are easier to troubleshoot.

A Terralith also impacts your developer experience as you must closely monitor who has access to the code to make changes. The merge conflicts in your VCS repository would turn into a massive headache if multiple people were making changes to the code base at the same time. Instead, developers prefer a self-service approach where they can pick and choose modules they need to use, create their Terraform configuration files from that, deploy the Terraform resources as they need, and manage the Terraform state.

How Scalr Helps with the Terralith

While Scalr won’t solve all issues around a Terralith, it helps provider guiding principles through the features. It also helps users who already have a Terralith to start breaking it down into manageable workspaces as they work to improve their code. Here are some key features:

Private Module Registry: As you refactor your Terralith into modules, Scalr's private module registry provides a centralized location to store, version, and share your modules. This helps users take the building blocks to create their own Terraform configuration files.

Remote State Management: Scalr offers remote state management. Even with a Terralith, remote state management helps prevent state file conflicts and provides a centralized location for your state, making collaboration easier and reducing the risk of state file loss or corruption.

Workspace Management: Scalr's workspace feature allows you to manage different Terraform deployments or components separately, even if they're part of a monolithic configuration. This can be a stepping stone towards modularization, as you can start separating the code at the workspace level before refactoring your code. Workspaces also promote self-service as there are access controls around who can create, update the workspaces, and execute a Terraform run from the workspace. A workspace has a one-to-one mapping with a single state file.

‍

Version Control Integration: By integrating with version control systems (VCS), such as Gitlab or Github, Scalr encourages better code organization and collaboration practices. Workspace owners can define working directories and trigger patterns to help focus on specific areas of code that should trigger a run.

Run Triggers: Run triggers in Scalr can help manage dependencies between different parts of your infrastructure, which can be particularly useful when breaking your Terralith into smaller, interconnected pieces. Run triggers will trigger a run in a downstream workspace based on an upstream workspace having a successful apply.

Policy as Code (OPA): Scalr's integration with OPA can help enforce best practices and prevent further growth of the Terralith by setting rules about resource creation, naming conventions, and other aspects of your Terraform configuration. OPA policies can define which modules can be used for specific Terraform resources.

Concurrent Runs: Scalr allows for concurrent runs, which can help mitigate some of the performance issues associated with large, monolithic configurations. After breaking down the Terralith, you might go from a single run to ten runs, but you don’t want them to run in a serial pattern. Scalr can support concurrent runs, helping you speed up your development cycles.

Variable Management: Centralized variable management in Scalr can help organize and control the numerous variables that often come with a Terralith, making it easier to refactor and modularize over time. These variables can be passed down to the modules that are consumed within the workspaces.

Team Management and Permissions: As you begin to break down your Terralith, Scalr's team management features allow you to control access to different parts of your infrastructure, encouraging separation of duty. This is the key to promoting self-service and improving your developer experience.

While these features can help manage a Terralith configuration, it's important to remember that they don't solve the fundamental issues of a monolithic architecture. The ultimate goal should still be to refactor the Terralith into a more modular, maintainable structure. Scalr can make this process smoother by providing a platform for collaboration, state management, and gradual refactoring.

Summary

Managing multiple Terraform modules that are all reusable modules will set your organization up for success in the long term. While it might seem easy to start with one module and continually add Terraform resources to it, you are only growing a compounding problem that will eventually be an issue in the future. Instead implement the best practices up front and explore using a product like Scalr or HashiCorp Terraform Cloud to guide you in to a long term scalable solution.

Try it out in Scalr today!