Versioning ARM Template deployments

When deploying resources on Azure with Azure Resource Manager you want to be in control of which resources are deployed and control their life span. To get the control you need to do deploy in a tested, standardized and reusable manner. This can be done by managing your resource creation as Infrastructure as Code.

Definition of Infrastructure as Code

Infrastructure as Code is the process of managing and provisioning computing infrastructure and its configuration through machine-processable definition files. It treats the infrastructure as a software system, applying software engineering practices to manage changes to the system in a structured, safe way.

Infrastructure as Code

The characteristics of Infrastructure as Code are:

• Declarative
• Single source of truth
• Increase repeatability and testability
• Decrease provisioning time
• Rely less on availability of persons to perform tasks
• Use proven software development practices for deploying infrastructure
• Idempotent provisioning and configuration

In this blog post, I will explain how to create Azure infrastructure with versioned ARM templates. For the deployments are the VSTS Build and Release pipelines used. The code or ARM templates are managed from a Git repository. Code in the Git repository can use the same practices as any other development project. By updating you code or templates you can deploy, upgrade or remove your infrastructure at any time. The Azure portal will no longer used to deploy your infrastructure. Everything needs to be deployed using your deployments pipelines. This will give you full traceability over what is deployed into your Azure environment.

Deployment pipelines for deploying versioned ARM templates
If you have a large number of infrastructure resources, it is good to know what the exact footprint is. If you know this, you can easily redeploy and make test environments without the always present questing: how much is this infrastructure the same as production? To get to the point you have that much control over your infrastructure, you can apply versioning over the deployments and their content. ARM templates in combination with VSTS can help you by this. When applying Infrastructure as Code this way you can test an actual infrastructure deployment and develop new templates at the same time. To do this you need two deployment pipelines:

• Deployment of the reusable ARM templates
• Pipeline for deploying the resource base on the reusable ARM templates

The templates used in the second pipeline are deployed in the first pipeline. These are linked called ARM templates. A linked ARM templates enable you to decompose the code that deploys your resources. The decomposed templates provide you the benefit of testing, reusing and readability. You can link to a single linked template or to a composed one that deploys many resources like a VM, or a complete set of PAAS resources.

Deployment pipeline for reusable linked ARM templates
The goal of this pipeline is to deploy a set of tested linked templates (a version) to a storage account from where they can be used. Every time you do an update to the templates (pull request). A deployment pipeline is triggered (continues integration) and when all tests are successful a new version is deployed and ready to use. The new deployments are side by side the earlier deployment. This way the actual resource deployments can use a specific version. The following figure gives an overview of the pipeline:

Build
The deployment starts with a pull request to the master branch of the Git repository. Then a new build is trigged. In this build pipeline, the sources are copied to a build artifact to be used in a release. Besides that, a build number is generated, that can be used as version number of the released templates.

Release
The release has several release steps to ensure that the templates are tested before it is published. In the sample, all steps are an automatic process. When the tests succeed the release continues with the deployment of the templates to the storage account where they can be used for the real infrastructure deployments.

Deploy test
The first step it to deploy to a test location on the storage account. This test location will be used to test the ARM templates. When you deploy the templates this can also help you in debugging errors by running a test deployment from your local machine. The only task in the environment is to do an Azure Blob File Copy. All the linked templates (the artifact) are deployed to the Azure storage account.

Test ARM templates
The second step the ARM templates can be tested. First you get a SAS token to access the storage account. The next step is to do a deployment of the ARM templates. This runs a test ARM template that tries to cover as many parameters as possible. The last step is to remove the resource group where you have deployed you test resources. If all steps succeed the templates are approved for release. You can have these steps multiple times for different types of resources and resource combination. When you do this step N times, you can run them in parallel. If you split this in multiple templates, it is also clear where you have a problem if the step fails.

Deploy the production version
The last step will deploy the templates to a location where the build number is used in the naming convention. The task Azure Blob File Copy is the same as in the first step, only the location where the files are copies is variable depending on the build number. This way the templates can be referenced by using the build number in the URL.

Pipeline for deploying the resource base on the reusable ARM templates
When all linked templates are deployed, they can be used to do the deployments of your Azure infrastructure. In the sample pipelines, I have only on test environment, the number of test environments can be different for each pipeline. One pipeline will deploys the resources of one Azure Resource group.

Build
The goal of the build is to produce an artifact of the templates that can be used in the release pipeline. The deployment starts with a pull request to the master branch of the Git repository. Then a new build is trigged. In this build pipeline, the sources are copied to a build artifact to be used in a release.

Release
The release pipeline will validate, test and then deploy the resource to the production environment. The templates in each environment are the same, the only difference are the parameter files. The parameter file can parameterize the sizes of resource deployed in the different environments. The sizes must be chosen wisely to represent the production environment, but keep in mind the costs of a running a test environment. In the main template, you keep a variable build, which you use to point to specific release from the previous pipeline. This way you can control the deployed version of you shared linked templates.

Validate templates
The first step is to validate the templates for all environments. This is done by running a deployment in Validation Mode. In Validation Mode the template is compiled, the template checked if it is syntactically correct and will be accepted by Azure Resource Manager. You have to do this for all environment to check if the parameter files are correct. When the step succeeds, the deployment to test will start.

Release test
This step has as goal to deploy and test the resource. If there is a need for a gatekeeper, approvals can be added at the beginning of this step. If not, the deployment of your test environment starts automatically. If possible use the Deployment Mode complete. This ensures you that the resources in the Azure Resource Group are the same as defined in the ARM template. The action consists of 2 tasks. First create a SAS token for the access to the Azure Storage. The next step does the actual deployment. If all succeeds you can optionally do some manual testing on the resource itself, or even add a script that does this.

Release production
The production release starts with a gatekeeper (approval). When you are content with the previous (test) resources, an approver can start the production deployment. All resources in the production resource group will be updated accordingly the ARM template. Try to run your deployment in Complete mode. Then you know that all resources in the resource group are the same as you defined in your Git repository. You are running an infrastructure as Code scenario.

Final thoughts
Setting up deployment pipelines for your ARM templates is an investment in front to give you control over the resources that are deployed in your Azure environment. When the pipelines are running, changes to your Azure environment are fully controlled from code. Al changes are traceable from VSTS. You have a Infrastructure as Code scenario.

Advertisement