Easy to Follow Terraform Bootstrap Strategy

Terraform is a powerful cloud infrastructure and resource management tool that teams often struggle with how to integrate. Many teams are also in different stages of their cloud development and may be undecided on using terraform at their particular developmental stage. The initial process documentation is often spread across resources in multiple training modules, user guides, Github, and more.

We are here to change that, providing a one-stop resource for a refined process to bootstrap any terraform solution plus initial core pieces of every terraform cloud solution. 

We have taken the search out for this widely used cloud infrastructure and resource management tool, giving all developers the ability to copy code quickly.

Below, I have provided details in an easy-to-use format. I have also personally vetted this process in large and small deployments. Once implemented, the solutions are ready for integration with a CI/CD system to be automatically executed whenever a code change occurs.

Local Bootstrap Phase

Create an initial ‘core’ solution. Initially the solution will be run locally from a developer’s workstation on the first execution. This provisions the cloud resources terraform requires for remote state management and to enable the terraform solution be run as part of a CICD pipeline job. After this one-time bootstrap phase is performed, the solution would then be executed in a CICD pipeline job.

Bootstrap Process

Create a local directory that is to be the ‘core’ solution that deploys the minimum cloud resources required by terraform.

Create a terraform.tf file and add the following

provider "azurerm" {
 version = "=2.40.0"
 features {}
 }
 terraform {
 // backend "azurerm" { }
 }

The ‘backend’ line is commented out for now and will be added in a later step.

Create an inputs.tf and add the following variables

variable "location" {
type = string
}

//Add any optional tags to be used in core

Create a core.tf file and add the following

locals {
core_rg_name = "<name of core rg>"
required_tags = {
createdby = "terraform"
/Additional optional tags
}
}

###################### RESOURCE GROUP(S) ######################
resource "azurerm_resource_group" "core_rg" {
name = local.core_rg_name
location = var.location
}

###################### STORAGE ACCOUNT(S) ######################
resource "azurerm_storage_account" "core_store" {
account_replication_type = "RAGRS"
account_tier = "Standard"
location = var.location
name = local.core_store_name
resource_group_name = azurerm_resource_group.core_rg.name
allow_blob_public_access = false

tags = local.required_tags
}

resource "azurerm_storage_container" "terraform_container" {
name = "terraform"
storage_account_name = azurerm_storage_account.core_store.name
container_access_type = "private"
}

If for some reason these resource must be created/managed by an external process/script, then it’s easy to replace them as ‘data’ elements so that terraform is still able to reference these resources for its purposes.

Outputs are very import to core as it enables one of the most important abstractions to take advantage of between the terraform core solution and the remaining solutions used to deploy resources (see remote state discussed in ‘infrastructure’ section)

outputs.tf

output "core_inputs" {
value = {
location = var.location
//Additional tags
}
}

output "core_rg" {
value = data.azurerm_resource_group.core_rg.name
}

output "ops_storage_account" {
value = {
name = azurerm_storage_account.core_store.name
key = azurerm_storage_account.core_store.primary_access_key
}
}

output "required_tags" {
value = local.required_tags
}

Now run the terraform commands to deploy the core solution

terraform init

terraform plan

terraform apply

Once the apply completes, the required resources to store the terraform state remotely are deployed.

Normal/Automated Phase of Solution Implementation

With the core state file being saved in cloud storage, terraform is now ready for ‘normal’ operations where multiple developers and automation tools can begin safely using the core solution without worry of overwriting changes as features like state locking are now enabled. The core solution is now ready to be extended with the addition of child solutions that can remotely reference the core state.

At runtime

Create a local-only core-config.tfvars file to configure the terraform backend for the core state file (do not commit this file!)

storage_account_name = "<use value from core output>"
container_name = "terraform"
key = "tfstate.core"
access_key = "<use value from core output>"

Remove the comment from terraform.tf

// backend "azurerm" { }
becomes
backend "azurerm" { }

Run terraform init again to configure the new backend and copy the state file that was created during the apply phase to this backend

terraform init -backend-config=core-config.tfvars

Accept the message displayed by terraform asking to copy the local state to the remote state store

The core solution is now shared and ready for integration into a CICD system and used by a team of engineers. This is also the ideal time to add additional resources that are good candidates for the core solution. Examples of this would include SPNs used by CICD and other services for access to cloud resources. ‘Global/Regional’ resources such as key stores, certificate stores. A VM for admin purposes and for hosting a CICD server. It’s important to include the name and additional attributes such as resource Id at a minimum in the outputs.tf for the core solution. This will become important for child solutions that utilize the resources created by the core solution.

Utilizing Core Remote State in Child Solutions

An essential abstraction to make use of in any cloud deployment using terraform is the terraform remote state resource. The core solution is intended to provide essential and common/reusable resources for use by other parts of the infrastructure. It’s important to abstract the core solution for multiple reasons.

• It’s important to utilize DRY and other software development best practices in terraform solutions to maintain quality and keep complexity low.
• Separation between critical resources and remaining infrastructure.
• Reduces risk of disruption of critical resources.
• Enables reuse of common/shared infrastructure components.
• Enables multiple developers to begin working in separate states, solutions, repos and terraform workspaces.

Create a new terraform solution folder (on the same level as the core folder, not a subfolder), as example we’ll name it ‘operations’

In operations, create a new terraform.tf

provider "azurerm" {    
features {}  
}

terraform {
backend "azurerm" { }
}

data "terraform_remote_state" "core" {
backend = "azurerm"
//workspace = terraform.workspace //Only use if workspaces are configured and used in core

config = {
container_name = var.rs_core_cn
storage_account_name = var.rs_core_san
key = var.rs_core_key
access_key = var.rs_access_key
}
}

The configuration is very similar to core with the addition of the remote state resource that requires four variable values to be populated. This creates the connection to the core remote state.

Create a new inputs.tf

variable "rs_access_key" {
description = "Storage account key used to access remote state file"
type = string
}

variable "rs_core_cn" {
description = "Remote state container_name"
default = "terraform"
type = string
}

variable "rs_core_key" {
description = "Remote state key"
default = "tfstate.core"
type = string
}

variable "rs_core_san" {
description = "Remote state storage account name"
type = string
}

Create a globals.auto.tfvars (auto loaded variable initialization) to set some static remote state variable values

//rs_core_cn = "terraform" //Use default value
//rs_core_key = "tfstate.core" //Use default value
rs_core_san = "<use value from core output>"

Create a operations.tf

data "azurerm_resource_group" "core" {
name = data.terraform_remote_state.core.outputs.core_rg
}

...

Any additional resources to be deployed in the operations solution would be added here. The example shown references the ‘core’ resource group created/referenced by the core solution then output for child solutions to easily reference instead of using a static  value.

(Optional) create outputs.tf

If there are any outputs that are useful for reference, they should be added here.

At runtime

Create a local-only operations-config.tfvars file to configure the terraform backend for the operations state file(do not commit this file!)

storage_account_name = "<use value from core output>"
container_name = "terraform"
key = "tfstate.operations"
access_key = "<used value from core output>"

Create a local-only terraform.tfvars file to autoload variable initialization (do not commit this file!)

rs_access_key = "<use value from core output>"

Operations is now ready to be applied

terraform init -backend-config=core-config.tfvars

terraform plan

terraform apply

Once the apply completes, the operations resources are deployed and the terraform state is remotely stored in the core storage account, but in a new state file named ‘tfstate.operations’.

Integrating with existing deployments

The design of a terraform solution will very closely match the deployment of the resources, not all of which are accessible to terraform in equal ways. For example, cloud permissions may allow for the deployment of instance-based resources such as VMs, DBs, file storage, etc. Still, they may not allow for creating or modifying users, groups, roles, networking, and certain other global/regional resources. To perform regular deployments, terraform may need to access those read-only resources.

---

ABOUT THE AUTHOR: Jared Darling

Jared Darling is a Quality Assurance Lead and Senior Consultant on the Application Development team, with nearly two decades of experience in quality assurance and software development. He is gifted in creating automation test frameworks and strategies, model-based testing, agile methodologies, and mentoring and leading teams in distributed development environments.

---