Levels Hierarchy | Azure Terraform SRE (2024)

To address enterprise complexity, we recommend isolating and decomposing a complex environment in multiple state files.

The main concern with state files is related to security, as a Terraform state file might contain credentials, security tokens, keys etc. It is reasonable that state files corresponding to different levels of privileges are stored separately.

Another concern with state file, which calls for segregation, is the speed of innovation: When someone is conducting a change for a resource in a given state file, no-one else can change the state file at the same time, so having a limited number of state files can hinder the testing or integration processes in the enterprise.

Finally since DevOps is building in teams, we want to unlock autonomy of teams yet delegating some functions (application teams will not re-invent the networking layers but build on it). It will mean the capability to read a state file to do rich composition but without the possibility to alter the state file from other teams.

In Azure Terraform SRE landing zones, separating the Terraform state files enables you to:

  • Control blast radius: if one configuration is deficient, it only impacts one landing zone and does not compromise the whole environment.
  • Enforce standard configuration: by using different Terraform state, you can compose a complex environment very fast.
  • Enable autonomy: different landing zones can enable innovation and features at different pace from each other.
  • Least privilege principles: Each level is protected by its own identity to prevent unwanted information disclosure and modifications.

The levels approach

We categorize and store state files into levels.

A level groups together a set of state files that needs separation because of different lifecycles or privileges.

The model can be summarized as below:

Levels Hierarchy | Azure Terraform SRE (1)Figure 1: An overview of the levels proposed by Azure Terraform SRE landing zones to structure Terraform states

The core principles are:

  • One level can host multiple state files: A level is implemented as an Azure storage account with specific permissions, you can store as many Terraform state files as necessary on a particular level.
  • Rover helps you locate the right storage account: Whether you are running CAF landing zones locally or in pipeline, rover helps you locate the right storage account for a given environment and level. Authorization to deploy at a level is granted by Azure RBAC model and enforced at runtime (with the current logged in user credentials, or via the pipeline credentials).
  • You can only write in your current level: at a given level, you can only write the Terraform state file that you are responsible for.
  • Read one level down: at any particular level, you are able to read information one level down for your composition (one AKS cluster at level 3 can read the Terraform state file at level 2 to compose).
  • Codeless state reading and enterprise composition: Azure Terraform SRE landing zones come with an object model that allow to load any landing zone (given principal accessing it has access ) in memory and compose from it.

Azure Terraform SRE for Azure proposes a hierarchy of landing zones based on five levels as described below:

Level 0: Core platform automation

The different landing zones represented in different state files at level 0 would typically be:

  • The launchpad (storage accounts, Key Vault, RBAC, etc.) components related to Terraform state management.
  • The subscription creation delegation capabilities derived from the Enterprise Agreement or Microsoft Customer Agreement.
  • The credential rotation mechanisms and role-based access control core models.

Level 1: Core platform governance

The different landing zones represented in different state files at level 1 would typically be:

  • The Azure landing zone (management groups, policies) this would include the core ALZ (formerly called enterprise-scale) capabilities with the related custom management group structures and policies configuration. You can either use the out-of-the-box settings for Azure LZ or customize the values for your organization.
  • The GitOps services for core platform automation: creating the pipelines, creating the Virtual Network and compute nodes (see below) required for the DevOps self-hosted agents.
  • The platform subscription creation for core platform (to create the core enterprise-scale subscriptions like Identify, Management, Connectivity etc.)
  • The management subscription services for core platform capabilities such as log management, Azure Monitor capabilities, etc.
  • The identity subscription services for core platform such as domain controller virtual machines, Azure Active Directory Domain Services, Azure AD Group mappings etc.

Level 2: Core platform connectivity

The different landing zones represented in different state files at level 2 would typically be:

  • The connectivity subscription components: including the virtual networking components like classic Virtual Network-based Hub and Spoke, Azure Virtual WAN, Azure Virtual WAN regional hub, site-to-site, point-to-site and ExpressRoute connectivity objects, or third parties Network Virtual Appliances. Due to their regional nature, it is likely that each of those components would live inside a different Terraform state file.
  • Additional identity subscription services: could be Azure Active Directory Domain Services (managed AD), or Virtual-machine-based domain controllers.
  • Additional management subscription services: could be System Center Configuration Manager, Shared Imaged Galleries, Monitor Alerts or third parties management tools.

Level 3: Application landing zones vending machine

A landing zone for application is split between two levels. Level 3 includes components of an application that are typically delegated to platform operations teams.

Capabilities in level 3 would be:

  • The application subscriptions creation: capability to create subscriptions for various environments: Development, Test, UAT, DR, etc.
  • The privileged infrastructure services: to support the application platform. This would usually include components like virtual network blocks, virtual network peering objects to hubs, route tables and delegated identities (to deploy level4 services and whenever applicable applications deployment credentials).

Level 4: Applications landing zone

This level contains the application configuration delegated for application teams.

It contains for example Azure Kubernetes Services Cluster, API Management services and all its dependency to deliver a service. This can be sourced from Azure landing zones solution accelerators, a custom solutions implemented using CAF module or any other Terraform code.

The following pictures illustrates the split between level 3 and 4:Levels Hierarchy | Azure Terraform SRE (2)Figure 2: Example of Data and AI landing zone for level 3 and 4

The broader picture on the complete environment would look like that:

Levels Hierarchy | Azure Terraform SRE (3)Figure 3: Subscription Vending Machine Example

Operating with levels

A deployment will typically contain:

  • A couple of level 0 landing zones.
  • Few level 1 and 2 landing zones.
  • Many level 3 landing zones.
  • Many level 4 applications will exist in an environment.

For a given level in the environment, each compute node (see below) will be assigned an identity (see below) that will be used to authenticate and authorize operations on:

  • The target Azure Subscription
  • The Terraform state file: will be Read and Write permissions for the current level, will be Read only permissions for a "lower" level type of landing zone, avoiding alterations on more privileged environments.

In the example above, each pipeline will have its lifecycle management:

  • Level 0 and 1 will be called when updating the core platform.
  • Level 2 will be triggered when you are opening a new regional hub, or adding connectivity capabilities.
  • Level 3 will be triggered when you deploy a new service (Application Gateway, App Service Environment, Azure Kubernetes Services, etc.)
  • Level 4 can be initiated as many times a day as you deploy code in your application environment.

An enterprise environment will consist of a series of pipelines enforcing the different types of environments in different subscriptions.

Levels Hierarchy | Azure Terraform SRE (2024)


What are different levels of Terraform? ›

Level 0 and 1 will be called when updating the core platform. Level 2 will be triggered when you are opening a new regional hub, or adding connectivity capabilities. Level 3 will be triggered when you deploy a new service (Application Gateway, App Service Environment, Azure Kubernetes Services, etc.)

What is the Azure equivalent of Terraform? ›

On Azure, three of the most obvious choices for Infrastructure as Code (IaC) are Bicep, Terraform, and Pulumi. Bicep is Microsoft's own domain-specific language, whereas Terraform is the open-source tool that is cloud agnostic.

What are the four stages of Terraform? ›

The workflows of Terraform are built on top of five key steps: Write, Init, Plan, Apply, and Destroy. Nevertheless, their details and actions vary between workflows.

Is Terraform easy or difficult? ›

How Difficult is the HashiCorp Terraform Certification Program. This exam is effortless to pass if one has the knowledge of Terraform basic understanding, the purpose why it is used, the workflow, function modules, and the workspace concepts. The below-mentioned are the best way to prepare for the exam.

Is there anything better than Terraform? ›

SaltStack. SaltStack is a Terraform alternative that offers hybrid, on-premises, and cloud infrastructure management. It uses Python to manage configurations and supports running commands remotely. It caters to both agentless and agent-based environments.

What is the AWS equivalent of Terraform? ›

What is the difference between Terraform and CloudFormation? Terraform and CloudFormation are both infrastructure-as-code (IaC) tools. CloudFormation is developed by AWS and only manages AWS resources. Terraform is developed by HashiCorp and can manage resources across a wide range of cloud vendors.

Why is Terraform better than Ansible? ›

Terraform sets up and manages your IT infrastructure, using an infrastructure as code approach. Ansible, on the other hand, focuses on automating IT tasks like provisioning and deployment. In short: Use Terraform for infrastructure setup and Ansible for configuration.

What is a 3 tier application using Terraform? ›

Three-tier architecture automation using Terraform
  1. Creates a VPC with the CIDR block provided in the region you want.
  2. Creates subnets for each layer.
  3. Creates an IGW and NAT gateway.
  4. Creates Route tables.
  5. Creates a RDS instance.
  6. Configures security group for Web layer.
  7. EC2 instances for webservers.

What are the Terraform versions? ›

Hashicorp Terraform
1.31 year and 9 months ago (21 Sep 2022)1.3.10 (13 Sep 2023)
1.22 years ago (18 May 2022)1.2.9 (07 Sep 2022)
1.12 years and 6 months ago (08 Dec 2021)1.1.9 (20 Apr 2022)
1.03 years ago (08 Jun 2021)1.0.11 (10 Nov 2021)
6 more rows
Jun 6, 2024

What are the three steps in Terraform? ›

Every resource that is managed by Terraform has a lifecycle, this lifecycle contains three stages; Apply (Create), Update, and Destroy.

How many types of modules are in Terraform? ›

Modules are used to create reusable components inside your infrastructure. There are primarily two types of modules depending on how they are written (root and child modules), and depending if they are published or not, we identify two different types as well (local and published).

Top Articles
Latest Posts
Article information

Author: Gov. Deandrea McKenzie

Last Updated:

Views: 6099

Rating: 4.6 / 5 (66 voted)

Reviews: 81% of readers found this page helpful

Author information

Name: Gov. Deandrea McKenzie

Birthday: 2001-01-17

Address: Suite 769 2454 Marsha Coves, Debbieton, MS 95002

Phone: +813077629322

Job: Real-Estate Executive

Hobby: Archery, Metal detecting, Kitesurfing, Genealogy, Kitesurfing, Calligraphy, Roller skating

Introduction: My name is Gov. Deandrea McKenzie, I am a spotless, clean, glamorous, sparkling, adventurous, nice, brainy person who loves writing and wants to share my knowledge and understanding with you.