Blog

MANAGING TERRAFORM CREDENTIALS

Managing variables effectively is one of the most important aspects of a fantastic IaC project. An IaC project can have multiple modules, all of which would use numerous resources and possibly have different values across various environments. To make this setup configurable, the project should be able to use variables and manage them effectively.

Let’s dive into understanding everything about the tfvars files and paving the way for creating the most awesome IaC projects.

What Are tfvars Files?

Tfvars files allow us to manage variable assignments systematically in a file with the extension .tfvars or .tfvars.json. Even though there are numerous ways to manage variables in Terraform, tfvars files are the best and most common way to do so due to their simplicity and effectiveness.

Knowing that the best way to learn new things is to do them ourselves, we’ll approach this handSetting up the base code

Let’s spin up a simple EC2 instance through Terraform.s-on to understand the various ways of managing variables in Terraform. In the code example below, we will set up a project to spin up an EC2 instance and learn about declaring and managing variables for it. There are code snippets as well as git repository code to refer to at each step.

Setting up the base code

Let’s spin up a simple EC2 instance through Terraform.

resource "aws_instance" "billing_server" {
 ami = data.aws_ami.ubuntu.id
 instance_type = "t2.micro"
 tags = {
   "service" = "billing"
 }
}

Check out the base code here.

Introducing variables

The base code works perfectly. However, imagine working in multiple environments, such as test, staging, and production. We would prefer to deploy a larger instance in production and a smaller instance in the test environment to save money. Meaning that the code remains the same except for the instance type, which varies depending on the environment.

We already know that variables can be used to solve this problem. To keep things running, we’ll create a variable for the instance type and set its default value to “t2.micro”.

variable "instance_type" {
 type = string
 default = "t2.micro"
 description = "EC2 instance type"
}





The variable can be referred to by using the var keyword followed by . and then the <variable_name>.





resource "aws_instance" "billing_server" {
 ami = data.aws_ami.ubuntu.id
 instance_type = var.instance_type
 tags = {
   		"service" = "billing"
 	}
}










Introducing-variable-github-code





The above code works but still relies on the default value. What if we want to change the instance type just for the production environment? How do we pass it a different value when deploying it to production?





There are numerous ways to accomplish this. Let’s look at them one by one.





    

  • Assigning a value through the command line.
    • 






terraform plan -var="instance_type=t2.large"















    

  • Using environment variables to set a value.
    • 






TF_VAR_instance_type="t2.small" terraform plan















Sure, both of these methods work. But imagine typing in values for 10 to 20 variables for 5–10 different environments! 🤯





Even if we save the command somewhere, consider how time-consuming simple changes like changing a few values would be.





This is where tfvars files come in handy. Terraform can load variable definitions from these files automatically. Let’s have a look.





Note: It’s not uncommon to have 10–20 variables in your IaC when deploying to multiple environments.





Tip: It is wise to break down the project into various small projects whenever the number of variables increases significantly.





Introducing tfvars files










Let’s assign the variable a value using the terraform.tfvars file:





    

  • We will start by creating a file named terraform.tfvars.
    • 




  • Next, we will assign the instance_type variable a value in this file.
    • 











Add-terraform-tfvars-file-github-code





    

  • Let us see if Terraform picks the assigned value in terraform.tfvars file when we run terraform plan. Remember the default instance_type variable value is t2.micro and value assigned in terraform.tfvars file is t2.large.
    • 











    

  • Notice that not only did Terraform load the value from the terraform.tfvars file, but it also overwrote the default value assigned to the variable.
    • 






Halfway there










Great! We solved the problem of manually writing large commands and making edits to those commands. But the other half of the problem remains, the problem of managing different versions of the variable values for different environments.





Some questions to think about:





    

  • Should we manually edit the tfvars file every time we deploy to a different environment?
    • 




  • Would this solution scale for 10–20 variables with frequent deployments to various environments?
    • 






Continuously updating the tfvars file may become tiresome.





What if we created multiple files like test.tfvars, staging.tfvars, production.tfvars, and then passed these variables into Terraform at runtime.





    

  • Let’s create two versions of the tfvars file by the name dev.tfvars and prod.tfvars
    • 




  • We will set the instance_type as t2.large for dev version and t2.xlarge for production.
    • 






Add-dev-prod-versions-github-code





    

  • Let’s try running terraform plan again and see which version terraform picks up by default. Take a guess before you run this command 😉
    • 











    

  • It picked neither of the versions we defined; it actually went with the t2.micro default value that is assigned to the variable.
    • 






How do we ask Terraform to use either of our versions then?





This can be done using the -var-file flag to specify the tfvars file to load. For instance, to load the production version, we will use the following command.





terraform plan -var-file=”prod.tfvars”















Awesome! We learned how to pass values to Terraform variables in a variety of ways and solved our many to many problems of managing multiple variables for many environments.





Autoloading tfvars file










Terraform auto loads tfvars files only when they are named terraform.tfvars or terraform.tfvars.json.





Is it possible to ensure that tfvars files following custom naming schemes are loaded automatically?





It is in fact possible to name your file whatever you wish and have Terraform load it automatically. All we have to do is provide the file name with .auto.tfvars as an extension. Let’s try it out.





We will rename the dev.tfvars file to dev.auto.tfvars and run terraform plan on it. The expectation is that Terraform should automatically pick the value t2.large from the dev.auto.tfvars file instead of the default value t2.micro.





Rename-dev-to-dev-auto-tfvars-github-code















We notice that Terraform automatically loads out our dev.auto.tfvars file as expected.





Terraform can load variable definitions from these files automatically if:





    

  • Files named exactly terraform.tfvars or terraform.tfvars.json.
    • 




  • Any files with names ending in .auto.tfvars or .auto.tfvars.json.
    • 






Variable Loading Precedence










An interesting thing to ponder is the precedence of the various methods of providing variable values.










Variable.tf Files










A typical Terraform project can have 10–20 variables. With this in mind, it becomes a priority to better manage the structure by logically separating all variables in a single place.





This way, we can easily declare and find variables in a single place.





Terraform recommends this single place be the variables.tf file.





Let us quickly extract the instance_type variable we created in the main.tf file into a new file named variable.tf file.





Move-all-variables-to-variable-tf-file-github-code










Remember that variable.tf is like any other Terraform file; there is nothing special about it, and it is only used as a way to logically separate variable declaration into a separate file.





terraform.tfvars vs variable.tf










Do not confuse terraform.tfvars files with variable.tf files. Both are completely different and serve different purposes.





    

  • variable.tf are files where all variables are declared; these might or might not have a default value.
    • 




  • variable.tfvars are files where the variables are provided/assigned a value.
    • 
















The difference becomes clear if we try assigning the variable a value that is not declared in the variable.tf file.















We notice that terraform raises a warning about assigning a value to an undeclared variable.





We conclude the difference as that the variables.tf just declare valid variables and optionally their types, and the tfvars file assigns them values.





💡 You might also like:









Child modules and hierarchy of processing










By default, Terraform only loads .tf present in the current directory. Any files in the subdirectories are not loaded. To load files from subdirectories, Terraform uses a module structure.





It’s vital to keep in mind that .tfvars values only apply to variables in the root module.





The root .tfvars files are essential and authoritative, while child .tfvars files have no effect.





Let us understand this by our code example.





    

  • Let us create a child module that spins another EC2 instance.
    • 






/*file path : ./modules/main.tf*/
resource "aws_instance" "billing_server" {
 ami = data.aws_ami.ubuntu.id
 instance_type = var.instance_type
 tags = {
   "service" = "billing"
 }
}










Create another variable.tf and terraform.tfvarsfiles inside the modules directory. Assign the instance_type variable a value as t2.small.





Add-child-module-to-show-hierarchy-github-code










    

  • Call the child module in the root module.
    • 






Call-child-module-in-root-module-github-code





    

  • Run terraform init for Terraform to load the module statically.
    • 




  • Run terraform plan.
    • 






We notice that the child module tfvars file is ignored and the instance type is set as t2.xlarge (root terraform.tfvars) instead of t2.small (child module terraform.tfvars).










Terragrunt to Keep Terraform Configuration DRY










Terragrunt is a tool outside Terraform that allows us to keep the Terraform configuration DRY and to reduce variable usage redundancies.





Terragrunt aims to minimize repetition, whether it’s in remote backends, multiple environments, or multiple variables per environment.





It employs a terragrunt.hcl file to centrally define a common template configuration and generate code from the defined template.





Check out our Terragrunt vs. Terraform comparison to learn more.





For a multi-environment use case, Terragrunt allows us to define our Terraform code once and to promote a versioned, immutable “artifact” of that same code from the environment to the environment. For instance, for the example we discussed, the Terragrunt structure would be as follows:





# infrastructure
├── prod
│   ├──main.tf
│   ├──terragrunt.hcl
├── test
│   ├──main.tf
│   ├──terragrunt.hcl
├── qa
│   ├──main.tf
│   ├──terragrunt.hcl
├── dev
│   ├──main.tf
     ├──terragrunt.hcl










Inputs










You can set values for your module’s input parameters by specifying an inputs block in terragrunt.hcl





inputs = {
	instance_type  = "t2.micro"
	instance_count = 10
	tags = {
		Name = "example-app"
	}
}










Whenever you run a Terragrunt command, Terragrunt will set any inputs you pass in as environment variables.





$ terragrunt apply
/* It is equivalent to:
TF_VAR_instance_type="t2.micro" \
TF_VAR_instance_count=10 \
TF_VAR_tags='{"Name":"example-app"}' \
terraform apply
*/










Using inputs, the Terragrunt file prod looks like this:





# infrastructure/prod/ec2/terragrunt.hcl
terraform {
  source = 
    "github.com:learn-tfvars/infrastructure-modules.git//ec2?ref=v0.0.1"
}
inputs = {
instance_type = "t2.xlarge"
}










The terragrunt.hcl file in the test environment will look similar, but it will configure smaller instances type to save money:





# infrastructure/test/ec2/terragrunt.hcl
terraform {
  source = 
    "github.com:learn-tfvars/infrastructure-modules.git//ec2?ref=v0.0.1"
}
inputs = {
instance_type = "t2.small"
}










When we run terragrunt apply, Terragrunt downloads the ec2 module into a temporary folder and passes the module with the input variables followed by the terraform apply command in that folder. This way, each module in each environment is defined by a single terragrunt.hcl file that solely specifies the Terraform module to deploy and the input variables specific to that environment.





Terragrunt is natively supported on Spacelift. By running your Terragrunt commands on Spacelift, you can extend the capabilities of Terragrunt with the powers of Spacelift. This will provide you access to other powerful features such as Spacelift Contexts and Policies. For example, using Spacelift Contexts, you can define a collection of variables and files and then attach these to Spacelift stack(s). Learn more here.





Key Points










Terraform provides several methods for declaring and assigning variables. Variable.tf and terraform.tfvars files are excellent for externalizing configurations and passing values so they can be easily deployed across multiple environments.





Terragrunt is a handy tool to keep terraform configuration DRY.





If you need more help with Terraform, check How to Automate Terraform Deployments blog post.





We encourage you also to explore how Spacelift makes it easy to work with Terraform. If you need any help managing your Terraform infrastructure, building more complex workflows based on Terraform, and managing AWS credentials per run, instead of using a static pair on your local machine, Spacelift is a great tool for this. It supports Git workflows, policy as code, programmatic configuration, context sharing, drift detection, and many more great features out of the box. Create a free account with Spacelift or book a demo with one of our engineers.





Note: New versions of Terraform will be placed under the BUSL license, but everything created before version 1.5.x stays open-source. OpenTofu is an open-source version of Terraform that will expand on Terraform’s existing concepts and offerings. It is a viable alternative to HashiCorp’s Terraform, being forked from Terraform version 1.5.6. OpenTofu retained all the features and functionalities that had made Terraform popular among developers while also introducing improvements and enhancements. OpenTofu works with your existing Terraform state file, so you won’t have any issues when you are migrating to it.