TV4's migration from Terraform Cloud to Scalr

Business Overview

TV4 and MTV3 are Sweden and Finland's largest commercial television channels, respectively. In addition to producing original programming and operating television channels, we offer two digital services each in the form of streaming services and news sites.

Technical Overview

The TV4 team manages the following providers with Terraform: AWS, GitHub, Akamai, Grafana, Kubernetes, and Helm. We also use AzureAD for SSO and SCIM provisioning to Scalr.

We migrated away from Terraform Cloud primarily due to cost concerns but also due to lackluster service performance.

Introduction

Like many others using Terraform Cloud, we were surprised by all of the changes going on at HashiCorp, including the announced pricing changes. Based on this we decided to move our 1,000 workspaces and 50,000 resources to a more cost-effective solution and wanted to share what our experience was like.

Discovery & POC

Our main requirement when selecting a product was to provide an alternative to Terraform Cloud that did not massively impact the day-to-day operations of our users. We wanted a change but did not want a change to be so impactful that it hindered our development experience after the migration. Scalr offers some of the same core functionality we were accustomed to:

• Ability to use the native Terraform CLI
• A Scalr workspace is the equivalent of a Terraform Cloud workspace
• A Scalr environment is very similar to a Terraform Cloud organization
• Scalr can be fully automated through their Terraform provider

We ended up verifying that the POC met our needs by migrating all of our (the platform team’s) workspaces a week ahead of time and dogfooding the service. 

The Migration from TFC to Scalr

We first had to determine the methods to migrate from Terraform Cloud to Scalr:

1. Allow application teams to migrate their own workspaces by giving them a deadline.
2. Go with the “big-bang” approach and completely automate it with a quick cutover.

We decided to go with a big-bang approach as we were comfortable with the migration testing that was done during the POC, and the app teams were bought in. We took the existing Scalr module and transformed it fully into Python, which we were more comfortable with. You can check out our version of this here.

When it was time to migrate, we froze infrastructure deployments for around 2 hours to perform this migration. It could have been faster if the Python script had used async/multithreading, but we figured it was an unnecessary optimization for a one-off task. 

We bumped into a couple of minor issues along the way regarding pagination, some IAM roles that were missing, and having to remove old TFC lockfiles, but these were all quickly resolved.

Overall, the migration was a large success, and since fully onboarding, there have been multiple comments from the app teams regarding the Scalr interface feeling more responsive and modern than TFC.

Improvements Since Moving to Scalr

Other than a better UI experience, we also experienced:

• Performance Improvements: Terraform Cloud was frustratingly slow at times when starting jobs after changes were pushed to GitHub, whereas Scalr has consistently been responsive and has started runs within a few seconds.
• Pricing: We are not locked into any billing plan. We decided to start month-to-month to make sure we were happy with the choice and then migrated to an annual plan for larger discounts when we were ready.
• Organizational Structure: The main benefit we saw from breaking the one large Terraform Cloud organization into multiple Scalr environments was that it’s much easier to assign credentials to workspaces and manage automation such as Slack notifications when changes are made to production workspaces.
• Helicopter views: The admin team now has more oversight into overall operations through the run dashboard at the account scope making it easier to identify issues in the pipeline.

This blog was published on behalf of TV4