Testing Arm Templates in Pipelines

One of the most important things with IaC is to test it. With ARM templates there are a few options as well.

To test your ARM Templates you could run the following:-

ARM-TTK can be used to lint and validate the arm templates.

it does the following:

• Validating the author’s intentions by eliminating unused parameters and variables,
• Applying security practices like highlighting if you have left a secret in plain text,
• Uses environment functions to provide constants like domain suffixes, rather than hard-coded values.

To install it locally, follow this article https://docs.microsoft.com/en-us/azure/azure-resource-manager/templates/test-toolkit

Then test your template with Test-AzTemplate -TemplatePath .\azuredeploy.json

This is useful for testing locally. But you can also install it in Azure DevOps to run as part of your pipeline. As you can see from the YAML file example below , the pipeline runs the ARM-TTK tests, any custom Pester tests then creates the resources, runs through the deployment twice, in case something fails on a redeploy. Finally and most importantly deletes the infrastructure, even if the other stages, steps, and jobs fail. It publishes the test results to the ADO.

- name: resource-group
  value: hub-spoke-$(Build.BuildId)
- name: location
  value: uksouth
- name: template-location
  value: '/solutions/azure-hub-spoke/*'
- name: template-name
  value: 'azuredeploy.json'
- name: pester-script-location
  value: '/tests/Test.ARMTemplate.ps1' 
- name: ttk-skip-test
  value: 'DependsOn-Best-Practices,Template-Should-Not-Contain-Blanks,apiVersions-Should-Be-Recent'

trigger:
  branches:
    include:
    - master
  paths:
    include:
      - /solutions/azure-hub-spoke/*
    exclude:
      - '/solutions/azure-hub-spoke/README.md'
      - '/solutions/azure-hub-spoke/bicep/*'
      
pr:
  branches:
    include:
    - master
  paths:
    include:
      - /solutions/azure-hub-spoke/*
    exclude:
      - '/solutions/azure-hub-spoke/README.md'
      - '/solutions/azure-hub-spoke/bicep/*'

schedules:
- cron: "0 12 * * 0"
  displayName: Weekly Sunday build
  branches:
    include:
    - master
  always: true
      
stages:

# Run ARM TTK and publish test results (Windows only)
- stage: armTemplateToolkit

  jobs:
  - job: armttk
    pool:
      vmImage: 'windows-latest'
    continueOnError: false
    timeoutInMinutes: 20

    steps:
    
    - task: PowerShell@2
      displayName: ARM-TTK and Pester
      inputs:
        targetType: 'inline'
        script: |
          git clone https://github.com/Azure/arm-ttk.git --quiet $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk
          import-module $env:BUILD_ARTIFACTSTAGINGDIRECTORY\arm-ttk\arm-ttk
          Install-Module Pester -AllowClobber -RequiredVersion 4.10.1 -Force -SkipPublisherCheck -AcceptLicense
          Import-Module Pester -RequiredVersion 4.10.1 -ErrorAction Stop
          $results = Invoke-Pester -Script @{Path = "$(System.DefaultWorkingDirectory)$(pester-script-location)"; Parameters = @{TemplatePath = "$(System.DefaultWorkingDirectory)$(template-location)$(template-name)"; Skip = "$(ttk-skip-test)"}} -OutputFormat NUnitXml -OutputFile TEST-ARMTemplate.xml -PassThru
          if ($results.TestResult.Result -contains "Failed") {Write-Error -Message "Test Failed"}
        
    - task: PublishTestResults@2
      inputs:
        testResultsFormat: 'NUnit'
        testResultsFiles: TEST-ARMTemplate.xml
      condition: always()

# Deploy template
- stage: validateAndDeploy
  dependsOn: []

  jobs:
  - job: arm
    pool: Hosted Ubuntu 1604
    continueOnError: false

    steps:    
    
    - task: AzureCLI@2
      displayName: Create resource group
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: 'az group create --name $(resource-group) --location $(location)'
               
    - task: AzureCLI@2
      displayName: Validate template (validation api)
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
         az deployment group validate --resource-group $(resource-group) --template-file $(System.DefaultWorkingDirectory)$(template-location)$(template-name) --parameters adminUserName=azureadmin adminPassword=$(adminPassword) linuxVMCount=1 windowsVMCount=1 deployVpnGateway=true
    - task: AzureCLI@2
      displayName: Deploy template
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
         az deployment group create --resource-group $(resource-group) --template-file $(System.DefaultWorkingDirectory)$(template-location)$(template-name) --parameters adminUserName=azureadmin adminPassword=$(adminPassword) linuxVMCount=1 windowsVMCount=1 deployVpnGateway=true
    - task: AzureCLI@2
      displayName: Deploy template (second pass)
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: |
         sleep 30
         az deployment group create --resource-group $(resource-group) --template-file $(System.DefaultWorkingDirectory)$(template-location)$(template-name) --parameters adminUserName=azureadmin adminPassword=$(adminPassword) linuxVMCount=1 windowsVMCount=1 deployVpnGateway=true
        
# Clean up deployment
- stage: cleanupResourceGroupBasic
  dependsOn: validateAndDeploy
  condition: always()

  jobs:
  - job: deleteResourceGroup
    pool: Hosted Ubuntu 1604
    continueOnError: false
    timeoutInMinutes: 20

    steps:

    - task: AzureCLI@2
      displayName: Delete resource group
      inputs:
        azureSubscription: $(serviceConnection)
        scriptType: 'bash'
        scriptLocation: 'inlineScript'
        inlineScript: 'az group delete --resource-group $(resource-group) --yes --no-wait'

Link to arm-ttk on github https://github.com/Azure/arm-ttk

Pester Tests

Pester is great for running customized tests. If you want to specifically target something or have a general test to cover all machines it is perfect. You can use it for Syntax/Linting, content validation, Template Validation and deployment validation. I’ve used it before to re-create the CIS standard to perform tests against entire subscriptions.

Describe "Tests" -tag "Infra" { 
    Context "ProjectA" { 
        $vNet=Get-AzVirtualNetwork -Name "$DeployedvNet" -ResourceGroupName $resourceGroup -ErrorAction SilentlyContinue 
            
        it "Check vNet is deployed $vNet " { 
            $vNet | Should Not be $null 
        } 
 

        it "Subnet $Deployedsubnet1 Should have Address Range 10.172.16.0/24" { 
            $subnet = Get-AzVirtualNetworkSubnetConfig -Name "$Deployedsubnet1" -VirtualNetwork $vNet -ErrorAction SilentlyContinue 
            $subnet.AddressPrefix | Should be "10.172.16.0/24" 
        } 

        it "Virtual Machine $vmName Should Be Size Standard_DS3_v4" {
            $vm.HardwareProfile.VmSize | should be "Standard_DS3_v4"
        }
    } 
}

AzTS and ADOScanner

Alongside these tests, I would also run AzTS (Azure Tenant Security Solution)which is the replacement for AZSK (Azure Security Kit). The AzTS gets deployed to your Azure Subscription and will run checks across multiple subscriptions.

The other option is to run the ADOScanner in a pipeline. This will check across your configuration in ADO. This can be installed from the marketplace into your ADO.

Hopefully this helps.