Azure - Infrastructure as a code - Part 1


What is Infrastructure as Code (IaC)

"Infrastructure as Code automates provisioning and management of cloud resources"

Creating modern cloud applications requires us to provision, configure and manage several cloud resources. Rather than configuring resources manually, we use descriptive templates, source code and Devops to eliminate manual configuration. IaC and Devops allows us to deploy and scale cloud applications with greater speed, less risk and with reduced cost.

"In IaC Infrastructure is treated with the same principles and practices as any other source code"

IaC generates the same environment every time it is applied and allows us to deliver stable environments rapidly and at scale. We can use all the same principles and practices that we use with any other source code (version control, automated testing, pull requests, code reviews)

"IaC removes the risk associated with human error, like manual misconfigurations"

Accidents happen, human error happens. Even experts make some once in a while. This is even more so if you have large teams and multiple people setting up and cofiguring your infrastructure. With IaC and devops practices we can reduce risks and deliver stable environments faster.


9 reasons to use IaC?

  • You can fully automate provisioning and management of cloud resources and entire environments
  • You can miminize risks by creating the same environment every time without user made mistakes
  • You can significantly improve productivity in large-scale environments
  • You can enforce consistency by representing the desired state of the environments
  • You can provision multiple environments reliably, at scale and on demand.
  • You can validate and test the IaC code to prevent deployment issues
  • You can manage infrastructure via source control and provide a detailed audit trail for changes.
  • You can use same quality gates that are used with source code (code reviews, versioning, pull requests, automated testing)
  • You can increase productivity and allow developers to spend more time executing higher-value tasks

Azure IaC with Pulumi, Terraform and ARM templates

In Azure we have several IaC tool options. Let's look how we can use some popular IaC tools to automate Azure infrastructure provisioning and configuration.

Let's look the following IaC tools

  • Azure CLI
  • ARM template with JSON
  • ARM template with Bicep
  • Terraform
  • Pulumi

Azure CLI scripts can be used to declaratively script cloud infrastructure. Azure CLI scripts can be used to provision and configure almost any Azure resource. The CLI scripts are simple to use and easy to read. Scripts are executed within either PowerShell or Bash. They're also straightforward to debug, especially when compared with ARM templates

Azure CLI scripts work well when you can tear down and redeploy your infrastructure. Updating and managing an existing environment is harder with Azure CLI. Many CLI commands aren't idempotent and that means they'll recreate the resource each time they're run, even if the resource already exists. Azure CLI is extremely useful in Devops tasks. You can use other IaC tools to manage most or all resources and use Azure CLI for testing or getting detailed information about deployed resources.

Azure Resource Manager (ARM) templates are native templating method in Azure. ARM is an API provisioning engine that is built into Azure and exposed as an API service. ARM enables you to deploy, update, delete, and manage Azure resources in a single, coordinated operation. You provide the engine with a JSON-based template that specifies the resources and their configuration. ARM automatically orchestrates the deployment in the correct order respecting dependencies. The engine ensures idempotency. If a desired resource already exists with the same configuration, provisioning will be ignored.

ARM templates are not the easiest to author. Now There is also a new Domain Specific Language (DSL) for ARM templates called Bicep.

HashiCorp Terraform is templating tool which can be used with all cloud providers. With Terraform you can you create, deploy, and manage infrastructure as code on any cloud. Terraform requires us to write programs in a custom domain-specific-language (DSL) called HashiCorp Configuration Language (HCL), and the Terraform engine takes care of provisioning and updating resources. HCL is human readable and easy to author.

Pulumi enables developers to write IaC in their favorite programming language, such as C#, TypeScript, JavaScript, Python, and Go. This enables us to create modern cloud applications and infrastructure without needing to learn a new configuration language. With Pulumi, we can use general purpose languages to express desired state of our infrastructure, and let Pulumi’s engine to give you diffs and a way to robustly update your infrastructure. Pulumi is also multicloud (Azure, AWS, Google). Because you’re using a full programming language, you get access to all the language features, like for loops, functions and classes.

Azure IaC examples

Let's see some Azure IaC examples using Azure CLI, ARM templates, Terraform and Pulumi

Azure CLI

Following Azure CLI script with Powershell creates resource group, SQL server and SQL database

$rgName="rg-iac-example"
$location="westeurope"
$serverName="sqlIaC"
$database="db"

az group create ` --name $rgName ` --location $location `
az sql server create ` --name $serverName ` --resource-group $rgName ` --location $location ` --admin-user $login ` --admin-password $password `
az sql db create ` --resource-group $rgName ` --server $serverName ` --name $database ` --edition GeneralPurpose ` --family Gen5 ` --capacity 2 ` --zone-redundant false

Azure CLI scripts are compact, easy to create and modify. For small projects CLI scipts can be usable. Some CLI commands aren't idempotent and that means they'll recreate the resource. Recreating resources is usually not possible in production environments. This it one of the main reasons CLI scripts have limited use in IaC.

Azure Resource Manager (ARM) JSON templates
Following ARM JSON template creates App Service Plan, App Service, SQL Server, SQL Database

{
  "$schema": "https://schema.management.azure.com/schemas/2019-04-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {
    "skuName": {
      "type": "string"
    },
    "skuCapacity": {
      "type": "int",
    },
    "sqlAdministratorLogin": {
      "type": "string",
    },
    "sqlAdministratorLoginPassword": {
      "type": "securestring"
    },
    "location": {
      "type": "string",
    }
  },
  "variables": {
    "hostingPlanName": "[concat('hostingplan', uniqueString(resourceGroup().id))]",
    "webSiteName": "[concat('webSite', uniqueString(resourceGroup().id))]",
    "sqlserverName": "[concat('sqlserver', uniqueString(resourceGroup().id))]",
    "databaseName": "sampledb"
  },
  "resources": [
    {
      "name": "[variables('sqlserverName')]",
      "type": "Microsoft.Sql/servers",
      "location": "[parameters('location')]",
      "apiVersion": "2020-02-02-preview",
      "properties": {
        "administratorLogin": "[parameters('sqlAdministratorLogin')]",
        "administratorLoginPassword": "[parameters('sqlAdministratorLoginPassword')]",
        "version": "12.0"
      },
      "resources": [
        {
          "name": "[variables('databaseName')]",
          "type": "databases",
          "location": "[parameters('location')]",
          "apiVersion": "2020-02-02-preview",
          "dependsOn": [
            "[variables('sqlserverName')]"
          ],
          "properties": {
            "edition": "Basic",
            "collation": "SQL_Latin1_General_CP1_CI_AS",
            "maxSizeBytes": "1073741824",
            "requestedServiceObjectiveName": "Basic"
          }
        }
      ]
    },
    {
      "apiVersion": "2019-08-01",
      "name": "[variables('hostingPlanName')]",
      "type": "Microsoft.Web/serverfarms",
      "location": "[parameters('location')]",
      "sku": {
        "name": "[parameters('skuName')]",
        "capacity": "[parameters('skuCapacity')]"
      },
      "properties": {
        "name": "[variables('hostingPlanName')]"
      }
    },
    {
      "apiVersion": "2019-08-01",
      "name": "[variables('webSiteName')]",
      "type": "Microsoft.Web/sites",
      "location": "[parameters('location')]",
      "dependsOn": [
        "[variables('hostingPlanName')]"
      ],
      "properties": {
        "name": "[variables('webSiteName')]",
        "serverFarmId": "[resourceId('Microsoft.Web/serverfarms', variables('hostingPlanName'))]"
      },
      "resources": [
        {
          "apiVersion": "2019-08-01",
          "type": "config",
          "name": "connectionstrings",
          "dependsOn": [
            "[variables('webSiteName')]"
          ],
          "properties": {
            "DefaultConnection": {
              "value": "[concat('Data Source=tcp:', reference(resourceId('Microsoft.Sql/servers/', variables('sqlserverName'))).fullyQualifiedDomainName, ',1433;Initial Catalog=', variables('databaseName'), ';User Id=', parameters('sqlAdministratorLogin'), '@', reference(resourceId('Microsoft.Sql/servers/', variables('sqlserverName'))).fullyQualifiedDomainName, ';Password=', parameters('sqlAdministratorLoginPassword'), ';')]",
              "type": "SQLAzure"
            }
          }
        }
      ]
    }
  ]
}

ARM templates in practice

ARM templates are native solution in Azure and all Azure resources are supported. ARM queries Azure APIs directly and there is no need for storing state like with other IaC tools (Terraform, Pulumi). Azure also provides deployment history which is not included with Terraform or Pulumi. One really helpful feature in Azure is that you can download ARM templates from almost any existing resource. In Azure portal you can also use Templates and Template Specs to validate your ARM templates.

ARM JSON templates are not easy to author. Mastering ARM JSON templates takes a lot of time and editing before you feel productive. ARM templates are not my preferred IaC solution in Azure as ARM JSON format is not for humans. I try to avoid using ARM JSON templates and use more human friendly DSL tools. ARM has also added preview so that you can see what changes will happen if you apply ARM template. You can use what-if operation to preview changes and decide if you want to apply those changes. This was previously missing and ARM is catching up with Terraform and Pulumi in features.

Next Generation ARM with Project Bicep There is also a new Domain Specific Language (DSL) for ARM templates called Bicep. Bicep DSL is closer to Terraform and much more readable and easier to author than JSON ARM templates. Bicep could be perfect solution in Azure projects in the future.

param serverName string = uniqueString('sqlServer')
param sqlDBName string = 'iacDB'
param location string = resourceGroup().location
param administratorLogin string
param administratorLoginPassword string {
  secure: true
}

resource server 'Microsoft.Sql/servers@2019-06-01-preview' = { name: serverName location: location properties: { administratorLogin: administratorLogin administratorLoginPassword: administratorLoginPassword } }
resource sqlDB 'Microsoft.Sql/servers/databases@2020-08-01-preview' = { name: '${server.name}/${sqlDBName}' location: location sku: { name: 'Standard' tier: 'Standard' } }

HashiCorp Terraform

Below terraform template will create resource group, SQL server and SQL database

resource "azurerm_resource_group" "example" {
  name     = "database-rg"
  location = "West Europe"
}

resource "azurerm_sql_server" "example" { name = "mssqlserver" resource_group_name = azurerm_resource_group.example.name location = azurerm_resource_group.example.location version = "12.0" administrator_login = "admin1234" administrator_login_password = "password1234!" }
resource "azurerm_mssql_database" "example" { name = "example-db" server_id = azurerm_sql_server.example.id collation = "SQL_Latin1_General_CP1_CI_AS" license_type = "LicenseIncluded" max_size_gb = 4 read_scale = true sku_name = "BC_Gen5_2" zone_redundant = true }

Terraform template and syntax is easy to read also for humans. You will need to install Terraform CLI. Terraform CLI enables you to validate and preview infrastructure changes before making changes to cloud resources.

Pulumi is similar to Terraform. You need to install Pulumi CLI. Pulumi enables you to preview changes and apply changes if changes are what you want to apply. Pulumi uses real programming languages (C#, Typescript, Python, Javascript, Go) and you can use constructs like classes, loops, arrays to define your infrastructure.

Following C# Pulumi code creates SQL server, SQL databases using provided List and an WebApp.

var sqlServer = new Server("sqlserver", new ServerArgs
{
    ResourceGroupName = resourceGroup.Name,
    AdministratorLogin = username,
    AdministratorLoginPassword = password,
    Version = "12.0",
});

foreach (var db in databases) { var database = new Database( $"{db.Name}", new DatabaseArgs { ResourceGroupName = resourceGroup.Name, ServerName = sqlServer.Name, Sku = new Pulumi.AzureNative.Sql.Inputs.SkuArgs { Name = "S0" } }); }
var app = new WebApp("app", new WebAppArgs { ResourceGroupName = resourceGroup.Name, ServerFarmId = appServicePlan.Id, SiteConfig = new SiteConfigArgs { ConnectionStrings = { new ConnStringInfoArgs { Name = "db", Type = ConnectionStringType.SQLAzure, ConnectionString = Output.Tuple(sqlServer.Name, database.Name, password).Apply(t => { (string server, string database, string pwd) = t; return $"Server= tcp:{server}.database.windows.net;initial catalog={database};userID={username};password={pwd};Min Pool Size=0;Max Pool Size=30;Persist Security Info=true;"; }), }, }, } }); }

Pulumi code is easy to follow and change. You get all the benefits of full programming language. You can use all the same programming and software architecture best practices that you would with application source code.

Final thoughts on Infrastructure as Code

"IaC and DevOps are essential when you build modern cloud applications"


Reasons to use IaC

  • Automation removes manual work and reduces mistakes
  • Speed increases when entire deployment process is fully automated from infrastructure management to application deployments
  • Safe, consistent, repeatable
  • Same principles can be applied as in any other source code process
  • Quality increases as IaC code is reviewed and automatically tested
  • IaC code is shareable and reusable in other project and between teams

Do you need help with IaC, Devops development?

We help to build IaC and devops solutions