How We Use Terraform At Slack

At Slack, we use Terraform for managing our Infrastructure, which runs on AWS, DigitalOcean, NS1, and GCP. Even though most of our infrastructure is running on AWS, we have chosen to use Terraform as opposed to using an AWS-native service such as CloudFormation so that we can use a single tool across all of our…

Slack logo
Slack
•
Archie Gunasekara
•17 min read•intermediate•
--
•View Original
AWSAWS EC2AWS S3ChefDigitalOceanDynamoDBGitGolangGroovyJenkinsTerraformYAML

Overview

The article discusses how Slack utilizes Terraform for managing its infrastructure across multiple cloud providers, including AWS, DigitalOcean, NS1, and GCP. It covers the evolution of Terraform state files, deployment mechanisms, version management, and module handling, providing insights into the challenges and solutions encountered during implementation.

What You'll Learn

1

How to manage Terraform state files effectively across multiple cloud providers

2

Why using a centralized Terraform management approach can streamline infrastructure deployment

3

How to implement a version management strategy for Terraform and its providers

4

When to utilize Jenkins for Terraform deployment pipelines

Prerequisites & Requirements

  • Understanding of Infrastructure as Code (IaC) principles
  • Familiarity with Terraform and Jenkins

Key Questions Answered

How does Slack manage its Terraform state files?
Slack manages its Terraform state files by organizing them per region in each child AWS account and maintaining separate state files for global services. This structure allows for better management of resources and faster deployment times by minimizing the number of resources in each state file.
What deployment mechanism does Slack use for Terraform changes?
Slack utilizes Jenkins as a deployment mechanism for Terraform changes. When a change is merged to a state file, it triggers a Jenkins pipeline that consists of planning and applying stages, ensuring that changes are validated before being applied to production environments.
What challenges did Slack face with its initial Terraform setup?
Initially, Slack faced challenges with managing a single AWS account that contained all services, leading to rate limits and access control difficulties. This prompted the decision to create dedicated AWS accounts for different teams and services, improving manageability and performance.
How does Slack handle Terraform version upgrades?
Slack handles Terraform version upgrades by deploying multiple versions of Terraform binaries and plugins. They introduced a version configuration file to manage which version to use based on the state file, allowing for smoother transitions between versions and minimizing breaking changes.

Key Statistics & Figures

Number of Terraform state files managed
1,400
Slack currently manages approximately 1,400 Terraform state files owned by different teams.

Technologies & Tools

Some links below are affiliate links. We may earn a commission if you make a purchase.

Infrastructure Management
Terraform
Used for managing infrastructure across multiple cloud providers.
CI/CD
Jenkins
Utilized for automating the deployment of Terraform changes.
Cloud Provider
AWS
Primary cloud provider for Slack's infrastructure.
Cloud Provider
Digitalocean
Secondary cloud provider used by Slack.
Cloud Provider
GCP
Another cloud provider utilized in Slack's infrastructure.

Key Actionable Insights

1
Implement a centralized management approach for Terraform state files to streamline deployments and reduce complexity.
By organizing state files per region and service, teams can manage resources more effectively, leading to faster deployment times and easier troubleshooting.
2
Utilize Jenkins for automating Terraform deployments to ensure consistent and repeatable infrastructure changes.
Automating the deployment process with Jenkins allows for better validation of changes and reduces the risk of human error during the deployment process.
3
Establish a version management strategy for Terraform and its providers to handle upgrades smoothly.
By managing multiple versions of Terraform and its providers, teams can avoid breaking changes and ensure compatibility across different state files.

Common Pitfalls

1
Failing to properly manage Terraform state files can lead to performance issues and deployment failures.
When state files are not organized effectively, it can result in longer deployment times and increased difficulty in managing resources, especially as the number of state files grows.

Related Concepts

Infrastructure As Code
Continuous Integration/Continuous Deployment
Cloud Management