I had an Azure DevOps YAML based pipeline that had been working but was now getting stuck with the message “Job is pending…” when trying to start a stage in which there is a deployment to an environment.

Looking at the logs and Azure DevOps UI it was not obvious what the issue was.

Solution

Turns out it was due to environment checks and approvals. There was a branch policy on the environment. This was set to only allow use of Azure DevOps Templates on a given branch. The edit that had been done to the YAML meant it was trying to extend a template in a branch that was not in the approved list.

As soon as the working branch was added to the proved list it all worked as expected.

So if you see “Job is pending” errors with no obvious reason, check the environment approvals and checks. Remember, any issues with these don’t show up in the build log

I have recently come across another edge case to add to the list of gotchas.

It seems you cannot use stagedependencies to access a variable declared in a deployment job i.e. when you are using an environment to get approval for a release.

The workaround is to add a job that is dependent on the deployment and set the custom variable within it. This variable can be accessed by a later stage as shown below

- stage: S1
  jobs:
  - deployment: D1
    strategy:
      runOnce:
        deploy:
          steps:
              - checkout: none
              - bash: echo "Can't access the variable if set in here"
  - job: J1
    dependsOn:
      D1
    steps:
      - checkout: none
      - bash: echo "##vso[task.setvariable variable=myvar;isOutput=true]True" 
        name: BashStep

- stage: S2
  condition: always()

  dependsOn: 
   - S1
  jobs:
   - job: Use_Variable
     variables: # add an alias for the var
       myvar: $[stagedependencies.S1.J1.outputs['BashStep.myvar']]
        steps:
          - checkout: none
          - dash: echo "Script gets run when myvar is true"
            condition: eq (variables['myvar'],'True')

I have been doing some work with Azure DevOps multi-stage YAML pipelines using stage dependency variables and conditions. They can get confusing quickly, you need one syntax in one place and another elsewhere.

So, here are a few things I have learnt…

What are stage dependency variables?

Stage Dependencies are the way you define which stage follows another in a multi-stage YAML pipeline. This is as opposed to just relying on the order they appear in the YAML file, the default order. Hence, they are critical to creating complex pipelines.

Stage Dependency variables are the way you can pass variables from one stage to another. Special handling is required, as you can’t just use the ordinary output variables (which are in effect environment variables on the agent) as you might within a job as there is no guarantee the stages and jobs are running on the same agent.

For stage dependency variables, is not how you create output variables, that does not differ from the standard manner, the difference is in how you retrieve them.

In my sample, I used a BASH script to set the output variable based on a parameter passed into the pipeline, but you can create output variables using scripts or tasks

 - stage: SetupStage
    displayName: 'Setup Stage'
    jobs:
      - job: SetupJob
        displayName: 'Setup Job'
        steps:
          - checkout: none
          - bash:  |
              set -e # need to avoid trailing " being added to the variable https://github.com/microsoft/azure-pipelines-tasks/issues/10331
              echo "##vso[task.setvariable variable=MyVar;isOutput=true]${{parameters.value}}"
            name: SetupStep
            displayName: 'Setup Step' 

Possible ways to access a stage dependency variable

There are two basic ways to access stage dependency variables, both using array objects

stageDependencies.STAGENAME.JOBNAME.outputs['STEPNAME.VARNAME']
dependencies.STAGENAME.outputs['JOBNAME.STEPNAME.VARNAME']

Which one you use, in which place, and whether via a local alias is the complexity

How to access a stage dependency in a script?

To access a stage dependency variable in a script, or a task, there are two key requirements

• The stage containing the consuming job and hence script/task, must be set as dependant on the stage that created the output variable
• You have to declare a local alias for the value in the stageDependencies array within the consuming stage. This local alias will be used as the local name by scripts and tasks

Once this is configured you can access the variable like any other local YAML variable

 - stage: Show_With_Dependancy
    displayName: ‘Show Stage With dependancy’
    dependsOn:
      - SetupStage
    variables:
      localMyVarViaStageDependancies : $[stageDependencies.SetupStage.SetupJob.outputs[‘SetupStep.MyVar’]]
    jobs:
      - job: Job
        displayName: ‘Show Job With dependancy’
        steps:
        - bash: |
              echo “localMyVarViaStageDependancies - $(localMyVarViaStageDependancies)” 

Tip: If you are having a problem with the value not being set for a stage dependency variable look in the pipeline execution log, at the job level, and check the ‘Job preparation parameters’ section to see what is being evaluated. This will show if you are using the wrong array object, or have a typo, as any incorrect declarations evaluate as null

How to use a stage dependency as a stage condition

You can use stage dependency variables as controlling conditions for running a stage. In this use-case you use the dependencies array and not the stagedependencies used when aliasing variables.

 - stage: Show_With_Dependancy_Condition
    condition: and (succeeded(), eq (dependencies.SetupStage.outputs['SetupJob.SetupStep.MyVar'], 'True'))
    displayName: 'Show Stage With dependancy Condition' 

From my experiments for this use-case, you don’t seem to need the DependsOn entry to decare the stage that exposed the output variable for this to work. So, this is very useful for complex pipelines where you want to skip a later stage based on a much earlier stage for which there is no direct dependency.

A side effect of using a stage condition is that many subsequent stages have to have their execution conditions edited as you cannot rely on the default completion stage state succeeded. This is because the prior stages could now be succeeded or skipped. Hence all following stages need to use the condition

condition: and( not(failed()), not(canceled()))

How to use a stage dependency as a job condition

To avoid the need to alter all the subsequent stage’s execution conditions you can set a condition at the job or task level. Unlike setting the condition at that stage level, you have to create a local alias (see above) and check the condition on that

 - stage: Show_With_Dependancy_Condition_Job
    displayName: 'Show Stage With dependancy Condition'
    dependsOn:
      - SetupStage
    variables:
      localMyVarViaStageDependancies : $[stageDependencies.SetupStage.SetupJob.outputs['SetupStep.MyVar']]
    jobs:
      - job: Job
        condition: and (succeeded(),
          eq (variables.localMyVarViaStageDependancies, 'True'))
        displayName: 'Show Job With dependancy' 

This technique will work for both Agent-based and Agent-Less (Server) jobs

A warning though, if your job makes use of an environment with a manual approval, the environment approval check is evaluated before the job condition. This is probably not what you are after, so if using conditions with environments that use manual approvals then the condition is probably best set at the stage level, with the knock-on issues of states of subsequent stages as mentioned above.

An alternative, if you are just using the environment for manual approval, is to look at using an AgentLess job with a manual approval. AgentLess job manual approvals are evaluated after the job condition, so do not suffer the same problem.

If you need to use a stage dependency variable in a later stage, as a job condition or script variable, but do not wish to add a direct dependency between the stages, you could consider ‘republishing’ the variable as an output of the intermedia stage(s)

 - stage: Intermediate_Stage
    dependsOn:
      - SetUpStage
    variables:
      localMyVarViaStageDependancies : $[stageDependencies.SetupStage.SetupJob.outputs['SetupStep.MyVar']]
    jobs:
      - job: RepublishMyVar
       steps:
          - checkout: none
          - bash:  |
              set -e # need to avoid trailing " being added to the variable https://github.com/microsoft/azure-pipelines-tasks/issues/10331
              echo "##vso[task.setvariable variable=MyVar;isOutput=true]$( localMyVarViaStageDependancies)"
            name: RepublishStep 

Summing Up

So I hope this post will help you, and the future me, navigate the complexities of stage variables

You can find the YAML for the test harness I have been using in this GitHub GIST

But what does my extension do?

As the Marketplace description says…

A tool to generate parameters.xml files for MSdeploy from the existing web.config file or from an app.config file for use with your own bespoke configuration transformation system.

Once the VSIX package is installed, to use right-click on a web.config, or app.config, file in Solution Explorer and the parameters.xml file will be generated using the current web.config entries from for both configuration/applicationSettings and configuration/AppSettings. The values attributes will contain TAG style entries suitable for replacement at deployment time.

I_f the parameters.xml already exists in the folder (even if it is not a file in the project) you will be prompted before it is overwritten._

Currently the version in the Marketplace of Parameter.xml Generator Tool supports Visual Studio 2015, 2017 & 2019

Adding Visual Studio 2022 Support

The process to add 2022 support is more complicated than adding past new versions, where all that was usually required was an update to the manifest. This is due to the move to 64Bit.

Luckily the process is fairly well documented, but of course I still had a few problems.

MSB4062: The “CompareBuildTaskVersion” task could not be loaded from the assembly

When I tried build the existing solution, without any changes, in Visual Studio 2022 I got the error

MSB4062: The “CompareBuildTaskVersion” task could not be loaded from the assembly D:myprojectpackagesMicrosoft.VSSDK.BuildTools.15.8.3253toolsVSSDKMicrosoft.VisualStudio.Sdk.BuildTasks.15.0.dll. Could not load file or assembly.

This was fixed by updating the package Microsoft.VSSDK.BuildTools from 15.1.192 to 16.9.1050.

Modernizing the Existing VSIX project

I did not modernize the existing VSIX project before I started the migration. When I clicked the Migrate packages.config to PackageReference…. it said my project was not a suitable version. So I just moved to the next step.

Adding Link Files

After creating the shared code project, that contains the bulk of the files, I needed to add links to some of the resources i.e. the license file, the package icon and .VSCT file.

When I tried add the link, I got an error in the form

_ Cannot add another link for the same file in another project_

I tried exiting Visual Studio, cleaning the solution, nothing helped. The solution was to edit the .CSPROJ file manually in a text editor e.g.

 <ItemGroup>
    <Content Include="ResourcesLicense.txt">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    <Content Include="..ParametersXmlAddinSharedResourcesPackage.ico">
      <Link>Package.ico</Link>
      <IncludeInVSIX>true</IncludeInVSIX>
    </Content>
    <Content Include="ResourcesPackage.ico">
      <CopyToOutputDirectory>Always</CopyToOutputDirectory>
    <Content Include="..ParametersXmlAddinSharedResourcesLicense.txt">
      <Link>License.txt</Link>
      <IncludeInVSIX>true</IncludeInVSIX>
    </Content>
    <EmbeddedResource Include="ResourcesParametersUppercaseTransform.xslt" />
    <VSCTCompile Include="..ParametersXmlAddinSharedParametersXmlAddin.vsct">
      <Link>ParametersXmlAddin.vsct</Link>
      <ResourceName>Menus.ctmenu</ResourceName>
    </VSCTCompile>
  </ItemGroup>

Publishing the new Extension

Once I had completed the migration steps, I had a pair of VSIX files. The previously existing one that supported Visual Studio 2015, 2017 & 2019 and the new Visual Studio 2022 version.

The migration notes say that in the future we will be able to upload both VSIX files to a single Marketplace entry and the Marketplace will sort out delivering the correct version.

Unfortunately, that feature is not available at present. So for now the new Visual Studio 2022 VSIX is published separately from the old one with a preview flag.

As soon as I can, I will merge the new VSIX into the old Marketpalce entry and removed the preview 2022 version of the VSIX

The use of variables is important in Azure DevOps pipelines, especially when using YML templates. They allow a single pipeline to be used for multiple branches/configurations etc.

The most common form of variables you see is are the predefined built in variables e.g. $(Build.BuildNumber) and your own custom ones e.g. $(var). Usually the value of these variables are set before/as the build is run, as an input condition.

But this is not the only way variables can be used. As noted in the documentation there are different ways to access a variable…

In a pipeline, template expression variables ${{ variables.var }} get processed at compile time, before runtime starts. Macro syntax variables $(var) get processed during runtime before a task runs. Runtime expressions $[variables.var] also get processed during runtime but were designed for use with conditions and expressions.

Azure DevOps Documentation

99% of the time I have been fine using just the $(var) syntax, but I recently was working on a case where this would not work for me.

The Issue

I had a pipeline that made heavy use of YML templates and conditional task insertion to include sets of task based upon the manually entered and pre-defined variables.

The problems that one of the tasks, used in a template, set a boolean output variable $(outVar) by calling

echo '##vso\[task.setvariable variable=outvar;isOutput=true\]true'

This task created the output variable could be accessed by other tasks as the variable $(mytask.outvar), but it was set at runtime it not available at the time of the YML compilation.

This caused me a problem as it meant that it could not be used in the template’s conditional task inclusion blocks as it as not present art compile time when this code is evaluated e.g.

\- ${{ if eq(mytask.outvar, 'true') }} :
  # the task to run if the condition is met
  - task: Some.Task@1 
    ....

I tied referencing the variable using all forms of $ followed by brackets syntax I could think of, but it did not help.

The lesson here is that you cannot make a runtime value a compile time value by wishing it to change.

The only solution I could find was to make use of the runtime variable in a place where it can be resolved. If you wish to enable or disable a task based on the variable value then the only option is to use the condition parameter

  # the task to run if the condition is met
  - task: Some.Task@1 
    condition: and(succeeded(), eq(mytask.outvar, 'true'))
    ....

The only downside of this way of working as opposed to the conditional insertion is that

• If you conditional insertion, non required tasks are never shown in the pipeline as they are not compiled into it
• If using the condition property to exclude a task, it will still appear in the log, but it can be seen that it has not been run.

So I got there in the end, it was just not as neat as I had hoped, but I do have a clearer understanding of compile and runtime variables in Azure DevOps YML

1. Edit the pipeline
2. Click the ellipse menu top right
3. Pick Triggers
4. Pick YAML and select the ‘Get Source’ block
5. Make sure the ‘Clean’ setting is set to ‘true’ and the right set of items to delete are selected – if this is not done the post clean up task does nothing
6. You can then add the post build cleanup task the end of the steps

steps:
  - script: echo This where you do stuff
  - task: mspremier.PostBuildCleanup.PostBuildCleanup-task.PostBuildCleanup@3
    displayName: 'Clean Agent Directories'
    condition: always()


```Once this is done it behaves as expected

This new version allows you to build release notes within a Multi-Stage YAML build since the last successful release to the current (or named) stage in the pipeline as opposed to just last fully successful build.

This gives more feature parity with the older UI based Releases functionality.

To enable this new feature you need to set the checkStage: true flag and potentially the overrideStageName: AnotherStage if you wish the comparison to compare against a stage other than the current one.

\- task: XplatGenerateReleaseNotes@3
  inputs:
    outputfile: '$(Build.ArtifactStagingDirectory)releasenotes.md'
    outputVariableName: 'outputvar'
    templateLocation: 'InLine'
    checkStage: true
    inlinetemplate: |
      # Notes for build 
      \*\*Build Number\*\*: {{buildDetails.id}}
      ...

Background

Back in the day I wrote a tool, TFS Alerts DSL, to do Work Item roll-up for TFS. Overtime I updated this to support VSTS (as Azure DevOps was then called), it’s final version is still available in the Azure DevOps Marketplace as the Azure DevOps Service Hooks DSL. So when I recently had a need for Work Item roll-up I did consider using my own tool, just for a short while. However, I quickly realised a much better option was to use the Aggregator CLI.

Aggregator CLI is a successor to the TFS Aggregator Plug-in and is a far more mature project than my tool and actively under development, allowing hosting as an Azure Function or a Docker container.

As I have found the Aggregator CLI a little hard to get started with, I thought this blog post was a good idea, so I don’t forget the details in the future.

Architecture

In this latest version of the Aggregator the functionality is delivered using Azure Functions, one per rule. Note: A docker container is an other option, but one I have not explored These Azure Functions are linked to Azure DevOps Service hook events. The command line tool setup process configures all of the parts required setting up Azure resources, Azure DevOps events and managing rules.

Preparation

• Download the latest release from https://github.com/tfsaggregator/aggregator-cli/releases, pick the version for the operating system you are planning to use to setup the tool.
• Next you need to setup an Azure Service Principle App registration for the Aggregator and connect it to a Subscription

1. 1. Login to Azure az login
   2. Pick the correct subscription az account set –subscription
   3. Create the service principle az ad sp create-for-rbac –name AggregatorServicePrincipal
   4. From the root of the Azure Portal pick the Subscription you wish to create the Azure Functions in.
   5. In the Access (IAM ) section grant the ‘contributor role’ for the subscription to the newly created Service Principle

Using the Aggregator CLI

At a command prompt we need to now start to use the tool to link up Azure Services and Azure DevOps

• First we log the CLI tool into Azure. You can find the values required from Azure Portal, in the Subscription overview and App Registration overview. You create a password from ‘client and secrets’ section for the App Registration. .aggregator-cli.exe logon.azure -s -c -t -p

• Next login to Azure DevOps, create the PAT as detailed in the documentation .aggregator-cli.exe logon.ado -u https://dev.azure.com/<org> -mode PAT -t

• Now we can create the Instance of the Aggregator in  Azure Note: I had long delays and timeout problems here due to what turned out to be a  poor WIFI links. The strange thing was it was not obviously failing WIFI, but just unstable enough to cause issues. As soon as I swapped to Ethernet the problems went away. The basic form of the install command is as follows, this will create a new resource group in Azure and then the required Web App, Storage, Application Insights etc. As this is  done using an ARM template so it is idempotent i.e. it can re run as many times as you wish, it will just update the Azure services if they already exist. .aggregator-cli.exe install.instance -verbose -n yourinstancename -l westeurope If you do get problems, goto the Azure Portal, find th reosurce group and look at the deployment logs

• When this completes, you can see the new resources in the Azure Portal, or check them with command line .aggregator-cli.exe list.instances

• You next need to register your rules. You can register as many as you wish. A few samples are provided in the test folder in the downloaded ZIP, these are good for a quick tests, thought you will usually create your own for production use. When you add a rule, behind the scenes this creates an Azure Function with the same name as the rule. .aggregator-cli.exe add.rule -v -i yourinstancename -n test1 -file testtest1.rule

• Finally you map a rule to some event in Azure DevOps instance .aggregator-cli.exe map.rule -v -p yourproject -e workitem.updated -i yourinstancename -r test1

Once all this done you should have a working system. If you are using the the test rules the quickest option to see it is working is to

1. Go into the Azure Portal
2. Find the created Resource Group
3. Pick the App Service for the Azure Functions
4. Pick the Function for the rule under test
5. Pick the Monitor
6. Pick Logs
7. Open Live Metric
8. You should see log entries when you perform the event on a work item you mapped to the function.

An alternative is to look in the AppInsights Logs or live telemetry. So I hope this helps my future self remember how get this tool setup quickly

However, running this is not as simple was you might expect so please read the instruction before proceeding.

Setup and Build

1. Clone the repo contain the Azure DevOps Extension.

2. Change to the folder

   _ExtensionsXplatGenerateReleaseNotesV2testconsole

   _

3. Build the tool using NPM (this does assume Node is already installed)

   _npm install
   npm run build

   _

Running the Tool

The task the testconsole runs takes many parameters, and reads runtime Azure DevOps environment variable. These have to be passing into the local tester. Given the number, and the fact that most probably won’t need to be altered, they are provided in settings JSON file. Samples are provided for a build and a release. For details on these parameters see the task documentation

The only values not stored in the JSON files are the PATs required to access the REST API. This reduces the chance of them being copied onto source control by mistake.

Two PATs are potentially used.

• Azure DevOps PAT (Required) - within a build or release this is automatically picked up. For this tool it must be provided
• GitHub PAT - this is an optional parameter for the task, you only need to provide it if working with private GitHub repos as your code store. So usually this can be ignored.

Test Template Generation for a Build

To run the tool against a build

1. In the settings file make sure the TeamFoundationCollectionUri, TeamProject and BuildID are set to the build you wish to run against, and that the ReleaseID is empty.

2. Run the command

   _node .GenerateReleaseNotesConsoleTester.js build-settings.json <Optional: your GitHub PAT>

   _

3. Assuming you are using the sample settings you should get an output.md file with your release notes.

Test Template Generation for a Release

To run the tool against a release is but more complex. This is because the logic looks back to see the most recent successful run. So if your release ran to completion you will get no notes as there has been no changes it it is the last successful release.

You have two options

• Allow a release  to trigger, but cancel it. You can then use its ReleaseID to compare with the last release
• Add a stage to your release this is skipped, only run on a manual request and use this as the comparison stage to look for difference

To run the tool

1. In the settings file make sure the TeamFoundationCollectionUri, TeamProject, BuildID, EnvironmentName (as stage in your process), ReleaseID and releaseDefinitionId are set for the release you wish to run against.

2. Run the command

   _node .GenerateReleaseNotesConsoleTester.js release-settings.json <Optional: yourGitHub PAT>

   _

3. Assuming you are using the sample settings you should get an output.md file with your release notes.

Hope you find it useful

To enable the feature, as it is off by default, you need to set the  getParentsAndChildren: true parameter for the task, either in YAML or in the handlebars section of the configuration.

This will add an extra array that the template can access relatedWorkItems. This contains all the work items associated with the build/release plus their direct parents and children. This can then be accessed in the template

{{#forEach this.workItems}}

{{#if isFirst}}### WorkItems {{/if}}

\* \*\*{{this.id}}\*\*  {{lookup this.fields 'System.Title'}}

\- \*\*WIT\*\* {{lookup this.fields 'System.WorkItemType'}} 

\- \*\*Tags\*\* {{lookup this.fields 'System.Tags'}}

\- \*\*Assigned\*\* {{#with (lookup this.fields 'System.AssignedTo')}} {{displayName}} {{/with}}

\- \*\*Description\*\* {{{lookup this.fields 'System.Description'}}}

\- \*\*Parents\*\*

{{#forEach this.relations}}

{{#if (contains this.attributes.name 'Parent')}}

{{#with (lookup\_a\_work\_item ../../relatedWorkItems  this.url)}}

      - {{this.id}} - {{lookup this.fields 'System.Title'}} 

{{/with}}

{{/if}}

{{/forEach}} 

\- \*\*Children\*\*

{{#forEach this.relations}}

{{#if (contains this.attributes.name 'Child')}}

{{#with (lookup\_a\_work\_item ../../relatedWorkItems  this.url)}}

      - {{this.id}} - {{lookup this.fields 'System.Title'}} 

{{/with}}

{{/if}}

{{/forEach}} 

{{/forEach}} 

This is a complex way to present the extra work items, but very flexible.

Hope people find the new feature useful.

In the past month or so I have added support for generating release notes based on PRs and Tests. I am now happy to say I have just added support for the actual files associated with a commit or changeset.

Enriching the commit/changeset data with the details of the files edited has been a repeated request over the years. The basic commit/changeset object only detailed the commit message and the author. With this new release of my task there is now a .changes property on the commit objects that exposes the details of the actual files in the commit/changeset.

This is used in Handlebars based template as follows

# Global list of CS ({{commits.length}})
{{#forEach commits}}
{{#if isFirst}}### Associated commits{{/if}}
* ** ID{{this.id}}** 
   -  **Message:** {{this.message}}
   -  **Commited by:** {{this.author.displayName}} 
   -  **FileCount:** {{this.changes.length}} 
{{#forEach this.changes}}
      -  **File path (use this for TFVC or TfsGit):** {{this.item.path}}  
      -  **File filename (using this for GitHub):** {{this.filename}}  
      -  **this will show all the properties available for file):** {{json this}}  
{{/forEach}}. 
{{/forEach}}

The new objects you can access are:

• In builds

• tests – all the test run as part of current build

• In releases

• tests – all the test run as part of any current build artefacts or previous to the running of the release notes task within a release environment

• releaseTests – all the test run within a release environment

• builds.test - all the test run as part of any build artefacts group by build artefact

These can be used as follows in a release template

\# Builds with associated WI/CS/Tests ({{builds.length}})

{{#forEach builds}}

{{#if isFirst}}## Builds {{/if}}

##  Build {{this.build.buildNumber}}

{{#forEach this.commits}}

{{#if isFirst}}### Commits {{/if}}

\- CS {{this.id}}

{{/forEach}}

{{#forEach this.workitems}}

{{#if isFirst}}### Workitems {{/if}}

\- WI {{this.id}}

{{/forEach}} 

{{#forEach this.tests}}

{{#if isFirst}}### Tests {{/if}}

\- Test {{this.id}} 

\-  Name: {{this.testCase.name}}

\-  Outcome: {{this.outcome}}

{{/forEach}} 

{{/forEach}}

  

\# Global list of tests ({{tests.length}})

{{#forEach tests}}

{{#if isFirst}}### Tests {{/if}}

\* \*\* ID{{this.id}}\*\* 

\-  Name: {{this.testCase.name}}

\-  Outcome: {{this.outcome}}

{{/forEach}}

  

For more details see the documentation in the WIKI

With the advent of cross-platform Powershell Core we realized that the current extension implementation had a fundamental limitation. Azure DevOps Tasks can only be executed by the agent using the Windows version of Powershell or Node. There is no option for execution by Powershell Core, and probably never will be. As Pester is now supported by Powershell Core this was a serious limitation.

To get around this problem I wrote a Node wrapper to allow the existing Powershell task to be executed using Node, by running a Node script then shelling out to Powershell or Powershell Core. A technique I have since used to make other extensions of mine cross-platform

Around this time we started to discuss whether my GitHub repo was really the best home for this Pester extension, and in the decided that this major update to provide cross-platform support was a good point to move it a new home under the ownership of Pester Project.

So, given all that history, I am really pleased to say that I am deprecating my Pester Extension and adding instructions that though my extension is not going away and will continue to work as it currently does, it will not be updated again and all users should consider swapping over to the new cross-platform version of the extension that is the next generation of same code base but now owned and maintained by the Pester project (well still Chris in reality).

Unfortunately, Azure DevOps provides no way to migrate ownership of an extension. So to swap to the new version will require some work. If you are using YAML the conversion is only a case of changing the task name/id. If you are using the UI based builds or release you need to add the new task and do some copy typing of parameters. The good news is that all the parameter options remain the same so it should be a quick job.

Also please note that any outstanding issues, not fixed in the new release, have been migrated over to the extensions now home, they have not been forgotten.

So hope you all like the new enhanced version of the Pester Extension and thanks to Chris for sorting the migration and all his work support it.

The Problem

I have a release that installs VSTest and runs some integration tests that target .NET 4.6 x64. All these tests worked fine in Visual Studio. However, I got the following errors for all tests when they were run in a release

2020-04-23T09:30:38.7544708Z vstest.console.exe "C:agent\_workr1aPaymentServicesdroptestartifactsPaymentService.IntegrationTests.dll"

2020-04-23T09:30:38.7545688Z /Settings:"C:agent\_work\_tempuxykzf03ik2.tmp.runsettings"

2020-04-23T09:30:38.7545808Z /Logger:"trx"

2020-04-23T09:30:38.7545937Z /TestAdapterPath:"C:agent\_workr1aPaymentServicesdroptestartifacts"

2020-04-23T09:30:39.2634578Z Starting test execution, please wait...

2020-04-23T09:30:39.4783658Z A total of 1 test files matched the specified pattern.

2020-04-23T09:30:40.8660112Z   X Can\_Get\_MIDs \[521ms\]

2020-04-23T09:30:40.8684249Z   Error Message:

2020-04-23T09:30:40.8684441Z    Test method PaymentServices.IntegrationTests.ControllerMIDTests.Can\_Get\_MIDs threw exception: 

2020-04-23T09:30:40.8684574Z System.BadImageFormatException: Could not load file or assembly 'PaymentServices, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' or one of its dependencies. An attempt was made to load a program with an incorrect format.

2020-04-23T09:30:40.8684766Z   Stack Trace:

2020-04-23T09:30:40.8684881Z       at PaymentServices.IntegrationTests.ControllerMIDTests.Can\_Get\_MIDs()

…

2020-04-23T09:30:40.9038788Z Results File: C:agent\_work\_tempTestResultssvc-devops\_SVRHQAPP027\_2020-04-23\_10\_30\_40.trx

2020-04-23T09:30:40.9080344Z Total tests: 22

2020-04-23T09:30:40.9082348Z      Failed: 22

2020-04-23T09:30:40.9134858Z ##\[error\]Test Run Failed.

Solution

I needed to tell vstest.console.exe to run x64 as opposed to it’s default of x32. This can be done with a command line override –platform:x64

[iframe width=“560” height=“315” src=“https://www.youtube.com/embed/WMQ0G9eXczE" frameborder=“0” allowfullscreen=”" allow=“accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture”]

The video up on YouTube

So your reporting objects are:

Array Objects

• workItems – the array of all work item associated with the release

• commits – the array of all commits associated with the release

• pullRequests - the array of all PRs referenced by the commits in the release

• The new one - builds - the array of the build artifacts that CS and WI are associated with. Note that this is a object with three properties

• build - the build details

• commits - the commits associated with this build

• workitems - the work items associated with the build

Release objects (only available in a release)

• releaseDetails – the release details of the release that the task was triggered for.
• compareReleaseDetails - the the previous successful release that comparisons are being made against

Build objects

• buildDetails – if running in a build, the build details of the build that the task is running in. If running in a release it is the build that triggered the release.

Note: To dump all possible values use the form {{json propertyToDump}} this runs a custom Handlebars extension to do the expansion

It is important to realised that these arrays are only available using the Handlebars form of templating. You can find sample here

The task now will, as well as checking for PR build trigger, try to associate the commits associated with a build/release pipeline to any completed PRs in the repo. This is done using the Last Merge Commit ID, and from my tests seems to work for the various types of PR e.g. squash, merge, rebased and semi-linear.

The resultant set of PRs are made available to the release notes template processor as an array. However, there is a difference between the existing arrays for Work Items and Commits and the new one for Pull requests. The new one is only available if you are using the new Handlebars based templating mode.

You would add a block in the general form….

{{#forEach pullRequests}}  
{{#if isFirst}}### Associated Pull Requests (only shown if  PR) {{/if}}  
\*  \*\*PR {{this.id}}\*\*  {{this.title}}  
{{/forEach}}

The reason I have chosen to only support Handlebars is it make the development so much easier and provides a more flexible solution, given all the Handlebar helpers available. I think this might be the first tentative step towards deprecating my legacy templating solution in favour of only shipping Handlebars support.

Well now I consider that multistage YAML pipelines are mature enough to allow me to do my whole release pipeline in YAML, hence this post.

My pipeline performs a number of stages, you can find a sample pipeline here. Note that I have made every effort to extract variables into variable groups to aid reuse of the pipeline definition. I have added documentation as to where variable are stored and what they are used for.

The stages are as follows

Build

The build phase does the following

• Updates all the TASK.JSON files so that the help text has the correct version number

• Calls a YAML template (build-Node-task) that performs all the tasks to transpile a TypeScript based task – if my extension contained multiple tasks this template would be called a number of time

• Get NPM packages

• Run Snyk to check for vulnerabilities – if any vulnerabilities are found the build fails

• Lint and Transpile the TypeScript – if any issue are found the build fails

• Run any Unit test and publish results – if any test fail the build fails

• Package up the task (remove dev dependencies)

• Download the TFX client

• Package up the Extension VSIX package and publish as a pipeline artifact.

Private

The private phase does the following

• Using another YAML template (publish-extension) publish the extension to the Azure DevOps Marketplace, but with flags so it is private and only assessible to my account for testing

• Download the TFX client

• Publishes the Extension to the Marketplace

This phase is done as a deployment job and is linked to an environment,. However, there are no special approval requirements are set on this environment. This is because I am happy for the release to be done to the private instance assuming the build phase complete without error.

Test

This is where the pipeline gets interesting. The test phase does the following

• Runs any integration tests. These could be anything dependant on the extension being deployed. Unfortunately there is no option at present in multistage pipeline for a manual task to say ‘do the manual tests’, but you could simulate similar by sending an email or the like.

The clever bit here is that I don’t want this stage to run until the new private version of the extension has been published and is available; there can be a delay between TFX saying the extension is published and it being downloadable by an agent. This can cause a problem in that you think you are running tests against a different version of the extension to one you have. To get around this problem I have implemented a check on the environment this stage’s deployment job is linked to. This check runs an Azure Function to check the version of the extension in the Marketplace. This is exactly the same Azure Function I already used in my UI based pipelines to perform the same job.

The only issue here is that this Azure Function is used as an exit gate in my UI based pipelines; to not allow the pipeline to exit the private stage until the extension is publish. I cannot do this in a multistage YAML pipeline as environment checks are only done on entry to the environment. This means I have had to use an extra Test stage to associate the entry check with. This was setup as follows

• Create a new environment

• Click the ellipse (…) and pick ‘approvals and checks’

• Add a new Azure Function check

• Provide the details, documented in my previous post, to link to your Azure Function. Note that you can, in the ’control options’ section of the configuration, link to a variable group. This is a good place to store all the values, you need to provide

• URL of the Azure Function

• Key to us the function

• The function header

• The body – this one is interesting. You need to provide the build number and the GUID of a task in the extension for my Azure Function. It would be really good if both of these could be picked up from the pipeline trying to use the environment. This would allow a single ‘test’ environment to be created for use by all my extensions, in the same way there are only a single ‘private’ and ‘public’ environment. However, there is a problem, the build number is picked up OK, but as far as I can see I cannot access custom pipeline variables, so cannot get the task GUID I need dynamically. I assume this is because this environment entry check is run outside of the pipeline. The only solution  can find is to place the task GUID as a hard coded value in the check declaration (or I suppose in the variable group). The downside of this is it means I have to have an environment dedicated to each extension, each with a different task GUID. Not perfect, but not too much of a problem

• In the Advanced check the check logic

• In control options link to the variable group contain any variables used.

Documentation

The documentation stage again uses a template (generate-wiki-docs) and does the following

• Use the extension and task manifest files to generate YAML usage documentation using one of my tasks
• Uploads the extension readme file to a WIKI using another of my tasks
• Uploads the extension YAML usage file to a WIKI

Public

The public stage is also a deployment job and linked to an environment. This environment has an approval set so I have to approve any release of the public version of the extension.

As well as doing the same as private stage this stage does the following

• Same a private stage
• Send a Tweet saying I have release a new version of the extension using my Artifact Description extension to get the text of the PR
• Update the pipelines minor build number variable to make sure then I next release I have a higher build number

Summary

It took a bit of trial and error to get this going, but I think I have a good solution now. The fact that the bulk of the work is done using shared templates means I should get good reuse of the work I have done. I am sure I will be able to improve the template as time goes on but it is a good start

I did not get to the bottom of why this is happening, but I found a fix.

• Check and of make a note of any UI declared variables in your Azure DevOps YAML Pipeline that is not triggering
• Delete the pipeline
• Re-add the pipeline, linking to the YAML file hosted on GitHub. You might be asked to re-authorise the link between Azure DevOps Pipelines and GitHub.
• Re-enter any variables that are declared via the Pipelines UI and save the changes

Your pipeline should start to be triggered again

This feature is limited, you obviously have to be using artifacts that linked to a Git repo, and also the Git repo have to on an Azure DevOps hosted repository. This won’t meet everyone’s needs but it is a start.

What was already available

Turns out there was already a means to get a limited set of PR details from a build. You used the form

\*\*Build Trigger PR Number\*\*: ${buildDetails.triggerInfo\['pr.number'\]}

or in handlebars format

\*\*Build Trigger PR Number\*\*: {{lookup buildDetails.triggerInfo 'pr.number'}} 

The improvements

That said I have improved the options. There is now a new `prDetails` object available to the template.

If you use the dump option

${JSON.stringify(prDetails)}      

You can see the fields available

{  
     "repository": {  
         "id": "bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59",  
         "name": "VSTSBuildTaskValidation",  
         "url": "[https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/\_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59"](https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59"),  
         "project": {  
             "id": "670b3a60-2021-47ab-a88b-d76ebd888a2f",  
             "name": "GitHub",  
             "description": "A container for GitHub CI/CD processes",  
             "url": "[https://richardfennell.visualstudio.com/\_apis/projects/670b3a60-2021-47ab-a88b-d76ebd888a2f"](https://richardfennell.visualstudio.com/_apis/projects/670b3a60-2021-47ab-a88b-d76ebd888a2f"),  
             "state": "wellFormed",  
             "revision": 411511726,  
             "visibility": 2,  
             "lastUpdateTime": "2019-10-10T20:35:51.85Z"  
         },  
         "size": 9373557,  
         "remoteUrl": "[https://richardfennell.visualstudio.com/DefaultCollection/GitHub/\_git/VSTSBuildTaskValidation"](https://richardfennell.visualstudio.com/DefaultCollection/GitHub/_git/VSTSBuildTaskValidation"),  
         "sshUrl": "richardfennell@vs-ssh.visualstudio.com:v3/richardfennell/GitHub/VSTSBuildTaskValidation",  
         "webUrl": "[https://richardfennell.visualstudio.com/DefaultCollection/GitHub/\_git/VSTSBuildTaskValidation"](https://richardfennell.visualstudio.com/DefaultCollection/GitHub/_git/VSTSBuildTaskValidation")  
     },  
     "pullRequestId": 4,  
     "codeReviewId": 4,  
     "status": 1,  
     "createdBy": {  
         "displayName": "Richard Fennell (Work MSA)",  
         "url": "[https://spsprodeus24.vssps.visualstudio.com/Ac0efb61e-a937-42a0-9658-649757d55d46/\_apis/Identities/b1fce0e9-fbf4-4202-bc09-a290def3e98b"](https://spsprodeus24.vssps.visualstudio.com/Ac0efb61e-a937-42a0-9658-649757d55d46/_apis/Identities/b1fce0e9-fbf4-4202-bc09-a290def3e98b"),  
         "\_links": {  
             "avatar": {  
                 "href": "[https://richardfennell.visualstudio.com/\_apis/GraphProfile/MemberAvatars/aad.NzQzY2UyODUtN2Q0Ny03YjNkLTk0ZGUtN2Q0YjA1ZGE5NDdj"](https://richardfennell.visualstudio.com/_apis/GraphProfile/MemberAvatars/aad.NzQzY2UyODUtN2Q0Ny03YjNkLTk0ZGUtN2Q0YjA1ZGE5NDdj")  
             }  
         },  
         "id": "b1fce0e9-fbf4-4202-bc09-a290def3e98b",  
         "uniqueName": "bm-richard.fennell@outlook.com",  
         "imageUrl": "[https://richardfennell.visualstudio.com/\_api/\_common/identityImage?id=b1fce0e9-fbf4-4202-bc09-a290def3e98b"](https://richardfennell.visualstudio.com/_api/_common/identityImage?id=b1fce0e9-fbf4-4202-bc09-a290def3e98b"),  
         "descriptor": "aad.NzQzY2UyODUtN2Q0Ny03YjNkLTk0ZGUtN2Q0YjA1ZGE5NDdj"  
     },  
     "creationDate": "2020-04-04T10:44:59.566Z",  
     "title": "Added test.txt",  
     "description": "Added test.txt",  
     "sourceRefName": "refs/heads/branch2",  
     "targetRefName": "refs/heads/master",  
     "mergeStatus": 3,  
     "isDraft": false,  
     "mergeId": "f76a6556-8b4f-44eb-945a-9350124f067b",  
     "lastMergeSourceCommit": {  
         "commitId": "f43fa4de163c3ee0b4f17b72a659eac0d307deb8",  
         "url": "[https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/\_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/f43fa4de163c3ee0b4f17b72a659eac0d307deb8"](https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/f43fa4de163c3ee0b4f17b72a659eac0d307deb8")  
     },  
     "lastMergeTargetCommit": {  
         "commitId": "829ab2326201c7a5d439771eef5a57f58f94897d",  
         "url": "[https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/\_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/829ab2326201c7a5d439771eef5a57f58f94897d"](https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/829ab2326201c7a5d439771eef5a57f58f94897d")  
     },  
     "lastMergeCommit": {  
         "commitId": "53f393cae4ee3b901bb69858c4ee86cc8b466d6f",  
         "author": {  
             "name": "Richard Fennell (Work MSA)",  
             "email": "bm-richard.fennell@outlook.com",  
             "date": "2020-04-04T10:44:59.000Z"  
         },  
         "committer": {  
             "name": "Richard Fennell (Work MSA)",  
             "email": "bm-richard.fennell@outlook.com",  
             "date": "2020-04-04T10:44:59.000Z"  
         },  
         "comment": "Merge pull request 4 from branch2 into master",  
         "url": "[https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/\_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/53f393cae4ee3b901bb69858c4ee86cc8b466d6f"](https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/commits/53f393cae4ee3b901bb69858c4ee86cc8b466d6f")  
     },  
     "reviewers": \[\],  
     "url": "[https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/\_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/pullRequests/4"](https://richardfennell.visualstudio.com/670b3a60-2021-47ab-a88b-d76ebd888a2f/_apis/git/repositories/bebd0ae2-405d-4c0a-b9c5-36ea94c1bf59/pullRequests/4"),  
     "supportsIterations": true,  
     "artifactId": "vstfs:///Git/PullRequestId/670b3a60-2021-47ab-a88b-d76ebd888a2f%2fbebd0ae2-405d-4c0a-b9c5-36ea94c1bf59%2f4"  
}  

```In templates this new object could be is used  

**PR Title **: ${prDetails.title}

or in handlebars format.

**PR Details**: {{prDetails.title}}

It will be interesting to here feedback from the real world as opposed to test harnesses

GitHub is a great system for individuals and OSS communities for both public and private project. However, corporate customers commonly want more control over their system than the standard GitHub offering. It is for this reason GitHub offers  GitHub Enterprise.

For most corporates, the essential feature that GitHub Enterprise offers is the use Single Sign On (SSO) i.e. allowing users to login to GitHub using their corporate directory accounts.

I wanted to see how easy this was to setup when you are using Azure Active Directory (AAD).

Luckily there is a step by step tutorial from Microsoft on how to set this up. Though, I would say that though detailed this tutorial has a strange structure in that it shows the default values not the correct values. Hence, the tutorial requires close reading, don’t just look at the pictures!

Even with close reading, I still hit a problem, all of my own making, as I went through this tutorial.

The Issue – a stray / in a URL

I entered all the AAD URLs and certs as instructed (or so I thought) by the tutorial into the Security page of GitHub Enterprise.

When I pressed the ‘Validate’ button in GitHub, to test the SSO settings, I got an error

‘The client has not listed any permissions for ‘AAD Graph’ in the requested permissions in the client’s application registration’

This sent me shown a rabbit hole looking at user permissions. That wasted a lot of time.

However, it turns out the issue was that I had a // in a URL when it should have been a  /. This was because I had made a cut and paste error when editing the tutorial’s sample URL and adding my organisation details.

Once I fixed this typo the validation worked, I was able to complete the setup and then I could to invite my AAD users to my GitHub Enterprise organisation.

Summary

So the summary is, if you follow the tutorial setting up SSO from AAD to GitHub Enterprise is easy enough to do, just be careful of over the detail.

Previous to this PR the templating in the task was done with a line by line evaluation of a template that used my own mark-up. This method worked but has limitations, mostly due to the line by line evaluation model.  With the Kenneth’s PR the option was added to write your templates in Handlebars, or stay with my previous templating engine.

Using Handlebars

If you use Handlebars, the template becomes something like

\## Notes for release  {{releaseDetails.releaseDefinition.name}}    
\*\*Release Number\*\*  : {{releaseDetails.name}}
\*\*Release completed\*\* : {{releaseDetails.modifiedOn}}     
\*\*Build Number\*\*: {{buildDetails.id}}
\*\*Compared Release Number\*\*  : {{compareReleaseDetails.name}}    

### Associated Work Items ({{workItems.length}})
{{#each workItems}}
\*  \*\*{{this.id}}\*\*  {{lookup this.fields 'System.Title'}}
   - \*\*WIT\*\* {{lookup this.fields 'System.WorkItemType'}} 
   - \*\*Tags\*\* {{lookup this.fields 'System.Tags'}}
{{/each}}

### Associated commits ({{commits.length}})
{{#each commits}}
\* \*\* ID{{this.id}}\*\* 
   -  \*\*Message:\*\* {{this.message}}
   -  \*\*Commited by:\*\* {{this.author.displayName}} 
{{/each}}

The whole template is evaluated by the Handlebars engine using its own mark-up to provide a means for looping across arrays and the like.

This seemed a great enhancement to the task. However, we soon realised that it could be better. Handlebars is extensible, so why not allow the extensibility to be used?

Using Handlebars Extensions

I have added extensibility in two ways. Firstly I have also added support for the common Handlebar-Helpers extensions, this added over 150 helpers. These are just accessed in a template as follows

\## To confirm the handbars-helpers is work
The year is {{year}} 
We can capitalize "foo bar baz" {{capitalizeAll "foo bar baz"}}

I have also added the ability to provide a block of JavaScript as a task parameter is that is loaded as a custom Handlebars extension. So if you add the following block in the tasks customHandlebarsExtensionCode parameter.

module.exports = {foo: function () {return 'Returns foo';}};

You can access in the templates as

\## To confirm our custom extension works
We can call our custom extension {{foo}}

It will be interesting to see how popular this alternative way of templating will be.

I recently tripped myself up whist adding SonarQube analysis to a rather complex Azure DevOps build.

The build has two VsTest steps, both were using the same folder for their test result files. When the first VsTest task ran it created the expected .TRX and .COVERAGE files and then published its results to Azure DevOps, but when the second VsTest task ran it over wrote this folder, deleting the files already present, before it generated and published it results.

This meant that the build itself had all the test results published, but when SonarQube looked for the files for analysis only the second set of test were present, so its analysis was incorrect.

Solution

The solution was easy, use different folders for each set of test results.

This gave me a build, the key items are shown below, where one VsTest step does not overwrite the previous results before they can be processed by any 3rd party tasks such as SonarQube.

steps:  
- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4  
   displayName: 'Prepare analysis on SonarQube'  
   inputs:  
     SonarQube: Sonarqube  
     projectKey: 'Services'  
     projectName: 'Services'  
     projectVersion: '$(major).$(minor)'  
     extraProperties: |  
      # Additional properties that will be passed to the scanner,   
      sonar.cs.vscoveragexml.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.coveragexml  
      sonar.cs.vstest.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.trx

  

… other build steps

  

\- task: VSTest@2  
   displayName: 'VsTest – Internal Services'  
   inputs:  
     testAssemblyVer2: |  
      \*\*\*.unittests.dll  
      !\*\*obj\*\*  
     searchFolder: '$(System.DefaultWorkingDirectory)/src/Services'  
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsServices'  
     overrideTestrunParameters: '-DeploymentEnabled false'  
     codeCoverageEnabled: true  
     testRunTitle: 'Services Unit Tests'  
     diagnosticsEnabled: True  
   continueOnError: true

\- task: VSTest@2  
   displayName: 'VsTest - External'  
   inputs:  
     testAssemblyVer2: |  
      \*\*\*.unittests.dll  
      !\*\*obj\*\*  
     searchFolder: '$(System.DefaultWorkingDirectory)/src/ExternalServices'  
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsExternalServices'  
     vsTestVersion: 15.0  
     codeCoverageEnabled: true  
     testRunTitle: 'External Services Unit Tests'  
     diagnosticsEnabled: True  
   continueOnError: true  

\- task: BlackMarble.CodeCoverage-Format-Convertor-Private.CodeCoverageFormatConvertor.CodeCoverage-Format-Convertor@1  
   displayName: 'CodeCoverage Format Convertor'  
   inputs:  
     ProjectDirectory: '$(System.DefaultWorkingDirectory)'  

\- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4  
   displayName: 'Run Code Analysis'

\- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4  
   displayName: 'Publish Quality Gate Result'  

As discussed in that post, using this model you will recreate your build agent VMs on a regular basis, as opposed to patching them. When I came to do this recently I found that the Packer image generation was failing with errors related to accessing packages.

Initially, I did not read the error message too closely and just assumed it was an intermittent issue as I had found you sometime get random timeouts with this process. However, when the problem did not go away after repeated retries I realised I had a more fundamental problem, so read the log properly!

Turns out the issue is you now have to pass a GitHub PAT token that has at least read access to the packages feed to allow Packer to authenticate with GitHub to read packages.

The process to create the required PAT is as follows

1. In a browser login to GitHub
2. Click your profile (top right)
3. Select Settings
4. Pick Developer Settings
5. Pick Personal Access Tokens and create a new one that has read:packages enabled

Once created, this PAT needs to be passed into Packer. If using the settings JSON file this is just another variable

{
"client\_id": "Azure Client ID",
"client\_secret": "Client Secret",
"tenant\_id": "Azure Tenant ID",
"subscription\_id": "Azure Sub ID",
"object\_id": "The object ID for the AAD SP",
"location": "Azure location to use",
"resource\_group": "Name of resource group that contains Storage Account",
"storage\_account": "Name of the storage account",
"ssh\_password": A password",
"install\_password": "A password",
"commit\_url": "A url to to be save in a text file on the VHD, usually the URL if commit VHD based on",

"github\_feed\_token": "A PAT"

}  

If you are running Packer within a build pipeline, as the other blog post discusses, then the PAT will be another build variable.

Once this change was made I was able to get Packer to run to completion, as expected.

There are times when you need to run Private Azure DevOps agents as opposed to using one of the hosted ones provided by Microsoft. This could be for a variety of reasons, including needing to access resources inside your corporate network or needing to have a special hardware specification or set of software installed on the agent.

If using such private agents, you really need to have an easy way to provision them. This is so that all your agents are standardised and easily re-creatable. Firstly you don’t want build agents with software on them you can’t remember installing or patching. This is just another form of the “works on one developer’s machine but not another” problem. Also if you have the means to replace the agents very regularly and reliably you can avoid the need to patch them; you can just replace them with newer VMs created off latest patched base Operating System images and software releases.

Microsoft uses Packer to build the VM images into Azure Storage. Luckily, Microsoft have open sourced their build tooling process and configuration, you can find the resources on GitHub. 

A fellow MVP, Wouter de Kort, has done an excellent series of posts on how to use these Packer tools to build your own Azure hosted Private Agents.

I don’t propose to go over that again. In this post, I will discuss what needs to be done to use these tools to create private agents on your own Hyper-V hardware.

By this point you are probably thinking ‘could this be done with containers? They are designed to allow the easy provisioning of things like agents’.

Well, the answer is yes that is an option. Microsoft provides both container and VM based agents and have only recently split the repo to separate the container creation logic from the VM creation logic. The container logic remains in the old GitHub home. However, in this post I am focusing on VMs, so will be working against the new home for the VM logic.

Preparation – Getting Ready to run Packer

Copy the Microsoft Repo

Microsoft’s needs are not ours, we wanted to make some small changes to the way that Packer builds VMs. The key changes are:

• We want to add some scripts to the repo to help automate our process.
• We don’t, at this time, make much use of Docker, so don’t bother to pre-cache the Docker images in the agent. This speeds up the image generation and keeps the VMs VHD smaller.

The way we manage these changes is to import the Microsoft repo into our Azure DevOps Services instance. We can keep our copy up to date by setting an upstream remote reference and from time to time merging in Microsoft’s changes, but more on that later.

All our are changes are done on our own long living-branch, we PR any revisions we make into this long lived branch.

The aim is to not alter the main Microsoft Packer JSON definition as sorting out a three way merge if both theirs and our versions of the main JSON file are updated is harder than I like. Rather if we don’t want a feature installed we add ’return $true’ at the start of the PowerShell script that installs the feature, thus allowing Packer to call the script, but skip the actions in the script without the need to edit the controlling JSON file.

This way of working allows us to update the master branch from the upstream repo to get the Microsoft changes, and then to regularly rebase our changes onto the updated master.

A local test of Packer

It is a good idea to test out the Packer build from a development PC to make sure you have all the Azure settings correct. This is done using a command along the lines of

packer.exe" build -var-file="azurepackersettings.json"  -on-error=ask "Windows2016-Azure.json"

Where the ‘windows2016-azure.json’ is the Packer definition and the ‘azurepackersettings.json’ the user configurations containing the following values. See the Packer documentation for more details

{
"client\_id": "Azure Client ID",
"client\_secret": "Client Secret",
"tenant\_id": "Azure Tenant ID",
"subscription\_id": "Azure Sub ID",
"object\_id": "The object ID for the AAD SP",
"location": "Azure location to use",
"resource\_group": "Name of resource group that contains Storage Account",
"storage\_account": "Name of the storage account",
"ssh\_password": A password",
"install\_password": "A password",
"commit\_url": "A url to to be save in a text file on the VHD, usually the URL if commit VHD based on"
}

If all goes well you should end up with a SysPrep’d VHD in your storage account after a few hours.

Note: You might wonder why we don’t try to build the VM locally straight onto our Hyper-V infrastructure. Packer does have a Hyper-V ISO builder but I could not get it working. Firstly finding an up to date patched Operative System ISO is not that easy and I wanted to avoid having to run Windows Update as this really slows the creation process . Also the process kept stalling as it could not seem to get a WinRM session, when I looked this seemed to be something to do with Hyper-V Vnet switches. In the end, I decided it was easier just to build to Azure storage. This also had the advantage of requiring fewer changes to the Microsoft Packer definitions, so making keeping our branch up to date easier.

Pipeline Process – Preparation Stages

The key aim was to automate the updating of the build images. So we aimed to do all the work required inside an Azure DevOps multistage pipeline. How you might choose to implement such a pipeline will depend on your needs, but I suspect it will follow a similar flow to ours.

1. Generate a Packer VHD
2. Copy the VHD locally
3. Create a new agent VM from the VHD
4. Repeat step 3. a few times

There is a ‘what comes first the chicken or the egg’ question here. How do we create the agent to run the agent creation on?

In our case, we have a special manually created agent that is run on the Hyper-V host that the new agents will be created on. This has some special configuration which I will discuss further below.

Stage 1 – Update our repo

As the pipeline has a source of our copy of the repo (and targets our branch), the pipeline will automatically get the latest version of our Packer configuration source in our repo. However, there is a very good chance Microsoft will have updated their upstream repo. We could of course manually update our repo as mentioned above and we do do this from time to time. However, just to make sure we are up to date, the pipeline also does a fetch, merge and rebases our branch on its local copy. To do this it does the following

1. Adds the Microsoft repo as an upstream remote
2. Fetches the latest upstream/master changes and merges them onto origin/master
3. Rebases our working branch onto the updated origin/master

Assuming this all works, and we have not messed up a commit so causing a 3-way merge that blocks the scripts, we should have all Microsoft’s latest settings e.g packages, base images etc. plus our customisation.

Stage 2 – Run Packer

Next, we need to run Packer to generate the VHD. Luckily there is a Packer extension in the Marketplace. This provides two tasks we use

1. Installs the Packer executable
2. Run Packer passing in all the values, stored securely as Azure DevOps pipeline variables, as used in the azurepackersettings.json file for a local test plus the details of an Azure subscription.

Being run within a pipeline has no effect on performance, so this stage is still slow, taking hours. However, once it is completed we don’t need to run it again so we have this stage set for conditional execution based on a pipeline variable so we can skip the step if it has already completed. Very useful for testing.

Stage 3 – Copy the VHD to a Local File Share

As we are building local private agents we need the VHD file stored locally i.e. copied down to a local UNC share. This is done with some PowerShell that runs the Azure CLI. It finds the newest VHD in the Azure Storage account and copies it locally, we do assume we are the only thing creating VHDs in the storage account and that the previous stage has just completed.

Again this is slow, it can take many hours depending on how fast your internet connection is. Once the VHD file is downloaded, we create a metadata file contains the name of profile it can be used with e.g. for a VS2017 or VS2019 agent and a calculated VHD file checksum, more details on both of these below.

Now again, as this stage is slow, and once it is completed we don’t need to run it again, we have conditional execution based on a second build variable so we can skip the step if it is not needed.

If all runs Ok, then at this point we have a local copy of a SysPre’d VHD. This can be considered the preparation phase over. These stages need to be completed only once for any given generation of an agent.

Pipeline Process – Deployment Stages

At this point we now have a SysPre’d VHD, but we don’t want to have to generate each agent by hand completing the post SysPrep mini setup and installing the Azure DevOps Agent.

To automate this configuration process we use Lability. This is a PowerShell tool that wrappers PowerShell’s Desired State Configuration (DSC). Our usage of Lability and the wrapper scripts we use are

discussed in this post by a colleague and fellow MVP Rik Hepworth. However, the short summary is that Lability allows you to create an ‘environment’ which can include one or more VMs. In our case, we have a single VM in our environment so the terms are interchangeable in this post.

Each VM in an environment is based on one or more master disk images. Each instance of a VM uses its own Hyper-V diff disk off their master disk, thus greatly reducing the disk space required. This is very useful when adding multiple virtually identical agent VMs to a single Hyper-V host.

A Lability environment allows us to have a definition of what a build VM is i.e. what is its base VHD image, how much memory does it have, are there any extra disks, how many of CPU cores does it have, this list goes on. Also, it allows us to install software, in our case the Azure DevOps agent.

All the Lability definitions are stored in a separate Git repo. We have to make sure the current Lability definitions are already installed along with the Lability tools on the Azure DevOps agent that will be running these stages of the deployment pipeline. We do this by hand  on our one ‘special agent’ but it could be automated.

Remember, in our case, this ‘special agent’ is actually domain-joined, unlike all the agents we are about to create, and running on the Hyper-V host where we will be deploying the new VMs. As it is domain joined it can get to the previously downloaded Sysprep’d VHD and metadata file on a network UNC share. We are not too worried over the ‘effort’ keeping the Lability definitions update as they very rarely change, all changes tend to be in the Packer generated base VHD.

It should be remembered that this part of the deployment is a repeatable process, but we don’t just want to keep registering more and more agents. Before we add a new generation agent we want to remove an old generation one. Hence, cycling old agents out of the system, keeping things tidy.

We have experimented with naming of Lability environments to make it easier to keep things tidy. Currently, we provide two parameters into our Lability configuration

• Prefix – A short-code to identify the role of the agent we use e.g. ‘B’ for generic build agents and ‘BT’ for ones with the base features plus BizTalk
• Index – This number is used for two jobs, the first is to identify the environment in a set of environments of the same Prefix. It is also used to work out which Hyper-V VNet the new environment should be attached to on the Hyper-V host. Lability automatically deals with the creation of these VNets if not present.

So for on our system, for example, a VM will end up with a name in the form B1BMAgent2019, this means

• B - It is a generic agent
• 1 – It is on the subnet 192.168.254.0, and is the first of the B group of agents
• BMAgent2019 – It is based on our VS2019 VHD image

Note: Also when an Azure DevOps Agent is registered with Azure DevOps, we also append a random number, based on the time, to the end of the agent name in Azure DevOps. This allows two VMs with the same prefix and index, but on different Hyper-V hosts, to be registered at the same time, or to have multiple agents on the same VM. In reality, we have not used this feature. We have ended up using unique prefix and index across agent our estate with a single agent per VM. 

Stage 4 – Disable the old agent then remove it

The first deployment step is done with a PowerShell script. We check to see if there is an agent registered with the current Prefix and Index. If there is we disable it via the Azure DevOps Rest API. This will not stop the current build but will stop the agent picking up a new one when the current one completes.

Once the agent is disabled we keep polling it, via the API, until we see it go idle. Once the agent is idle we can use the Azure DevOps API to delete the agent’s registration on Azure DevOps.

Stage 5 – Remove the old Environment

Once the agent is no longer registered with Azure DevOps we can then remove the environment running the agent. This is a Lability command that we wrapper in PowerShell scripts

This completely removes the Hyper-V VM and its diff disks that store its data, a very tidy process.

Stage 6 – Update Lability Settings

I said previously that we rarely need to update the Lability definitions. There is one exception, that is the reference to the base VHD. We need to update this to point to the copy of the Packer generated SysPrep’d VHD on the local UNC file share.

We use another PowerShell script to handle this. It scans the UNC share for metadata files to find the one containing the request media type e.g. VS2017 or VS2019 (we only keep one of each type there). It then registers this VHD in Lability using the VHD file path and the previously calculated checksum. Lability uses the checksum to work out if the file has been updated.

Stage 7 - Deploy New Environment

So all we have to do at this point is request Lability to create a new environment based on the variable parameters we pass in i.e. environment definition, prefix and any override parameters (the Azure DevOps Agent configuration) into a wrapper script.

When this script is run, it triggers Lability to create a new VM using an environment configuration.

Lability’s first step is to create the VNet if not already present.

It then checks, using the checksum, if the base Sysprep’d VHD has been copied to the Hyper-V host. If it has not been copied it is done before continuing. This can take a while but is only done once.

Next, the environment (our agent VM) is created, firstly the VM settings are set e.g. CPU & Memory and then the Windows mini setup is handled by DSC. This sets the following

• Administrator user Account and Password
• Networking, here we have to rename an Ethernet adapter. We have seen the name of the first Ethernet change across different versions of the Packer image, so to make our lives easier we rename the primary adaptor connected to the VNet to a known value.
• Swap Disk, set this allow the Operating System to manage this as the default on the Packer image is to use a dedicated drive D: which we don’t have.
• Create a dedicated drive E: for the agent.
• Download, install and configure the Azure DevOps agent

DSC handles any reboots required.

After a few minutes, you should see a new registered agent in the requested Azure DevOps Agent Pool.

Stage 8 – Add Capabilities

Our builds make use of Azure DevOps user capabilities to target them to the correct type of agent. We use an yet another PowerShell script that waits until the new agent been registered and then it adds our custom capabilities from a comma-separated string parameter.

A little tip here. A side effect of our Lability configuration is that all the agents have the same machine name. This can make finding the host they are on awkward, especially if you have a few Hyper-V hosts. So to address this problem we add a capability of the Hyper-V hosts name, this is purely to make finding the VM easier if we have to.

Stage  9 – Copy Capabilities

We have seen that some of the Azure DevOps tasks we use have demands that are not met by the System Capabilities. The classic is a task requiring a value for the capability ‘java’ or ‘jdk’ when  the one that is present on the agent is ‘JAVA_HOME’.

To address this, as opposed to adding our own capability which might not point to the correct location, is to copy an existing capability that has the correct value. Again this is done with a PowerShell script that takes as string parameter

So what do we end up with?

When all this completes we have a running private agent that has all the features of the Microsoft hosted ones. As Microsoft adds new functionality or patch their agent images, as long as we regenerate our Packer images, we get the same features.

At this point in time, we have chosen to add any extra software we require after the end of this process, as opposed to within it. In our case, this is basically either BizTalk 2013 and BizTalk 2016 on a single agent in our pool. Again we do this with a series of scripts, but manually run this time. We would like to fully automate the process, but BizTalk does not lend itself to easy installation automation. So, after a good bit of experimentation, we decided the best option for now, was to keep our basic build process as close to the Microsoft Packer images as possible to minimise merge issues, and worry about BizTalk later. As we only have one BizTalk 2013 and one 2016 agent the cost of manually finishing off was not too high.

Where do we go from here?

We now have a process that is automated end to end. However, it can be ‘a little brittle’, but as all the stages tidy up after themselves rerunning jobs is not an issue other than in the time cost.

We still have not decided on a final workflow for the replacement of agent. At this time we use manual approvals before deploying an agent. I am sure this will change as we allow this process to mature.

It is a good starting point.

Added Support for Builds

Prior to version 2.17.x this extension could only be used in Releases. This was because it used Release specific calls provided in the Microsoft API to work out the work items and changesets/commits associated with the Release. This is unlike my older PowerShell based Release Notes Extension which was initially developed for Builds and only later enhanced to  work in Releases, but achieved this using my own logic to iterate across Builds associated with Releases to work out the associations. With the advent of YAML multistage Pipelines the difference between a Build and a Release is blurring, so I thought it high time to add Build support to my Cross Platform Release Notes Extension. Which it now does.

Adding Tag Filters

In the Cross Platform Release Notes Extension you have been able to filter the work items returned in a generated document for a good while, but the filter was limited to a logical AND i.e. if the filter was

@@WILOOP:TAG1:TAG2@@

All work items matched would have to have both the TAG1 and TAG2 set Since 2.18.x there is now the option of a logic AND or an OR.

• @@WILOOP:TAG1:TAG2@@ matches work items that have all tags (legacy behaviour for backward compatibility)
• @@WILOOP[ALL]:TAG1:TAG2@@ matches work items that have all tags
• @@WILOOP[ANY]:TAG1:TAG2@@ matches work items that any of the tags

Update 5th Dec In 2.19.x there is also the option to filter on any field in a work item as well as tags

• @@WILOOP[ALL]:System.Title=This is a title:TAG 1@@

For more details see the extension’s WIKI page

Futures

My plan is to at some point deprecate my PowerShell based Release Notes Extension. I have updated the documentation for this older extension state as much and to recommend the use of the newer Cross Platform Release Notes Extension. At this time there is now little that this older extension can do that cannot be done by my newer Cross Platform Release Notes Extension. Moving to it I think makes sense for everyone for the…

• Cross platform support
• Use of the same means to find the associated items as the Microsoft UI to avoid confusion
• Enhanced work item filtering

Lets see of the new features and updated advisory documentation effect the tow extension relative download statistics

The Problem

The error occurred when I tried to queue a new build after the upgrade, the build instantly failed with the error

‘The module being executed is not trusted, Either the owner of the database of the module need to be granted authenticate permission, or the module needs to be digitally signed. Warning: Null value is eliminated by an aggregate or other SET operation, The statement has been terminated’.

The Solution

It turns out the issue was the the client was using a enterprise wide SQL cluster to host the tfs_ databases. After the Azure DevOps upgrade the DBAs has enabled a trigger based logging system to monitor the databases and this was causing the error.

As soon as this logging was switched off everything worked as expected.

I would not recommend using such a logging tool for any ‘out the box’ database for a product such as TFS/Azure DevOps Server where the DBA team don’t own the database schema’s changes, as these databases will only occur if the product is upgraded

To do a WI update you need to call the REST API

• Using the verb PATCH
• With the Header “Content-Type” set to “application/json-patch+json”
• Include in the Body the current WI update revision (to make sure you are updating the current version)

So the first step is to get the current WI values to find the current revision.

So my update logic was along the lines of

1. Create new WebClient with the Header “Content-Type” set to “application/json-patch+json”
2. Do a Get call to API to get the current work item
3. Build the update payload with my updated fields and the current revision.
4. Do a PATCH call to API using the client created in step 1 to update the current work item

Problem was at Step 4 I got a 400 error. A general error, not too helpful

After much debugging I spotted the issue was that after Step 2. my WebClient’s Headers had changed, I had lost the content type – no idea why.

It all started to work if I recreated my WebClient after Step 2, so something like (in PowrShell)

Function Get-WebClient

{

param  
(

> \[string\]$pat,  
> \[string\]$ContentType = "application/json"

)

> $wc = New-Object System.Net.WebClient  
> $pair = ":${password}"  
> $bytes = \[System.Text.Encoding\]::ASCII.GetBytes($pair)  
> $base64 = \[System.Convert\]::ToBase64String($bytes)  
> $wc.Headers.Add(“Authorization”,"Basic $base64")  
> $wc.Headers\["Content-Type"\] = $ContentType  
> $wc

}

  

function Update-WorkItemTitle {

param

(

$baseUri ,
$teamproject,
$workItemID,
$pat,
$title

)

$wc = Get-WebClient -pat $pat -ContentType “application/json-patch+json”
$uri = “$($baseUri)/$teamproject/_apis/wit/workitems/$($workItemID)?api-version=5.1”

# you can only update a work item if you also pass in the rev, this makes sure you are updating lastest version
$jsondata = $wc.DownloadString($uri) | ConvertFrom-Json

$wc = Get-WebClient -pat $pat -ContentType “application/json-patch+json”

$data = @(
@{
            op    = “test”;
            path  = “/rev”;
            value = $jsondata.Rev
},
@{
            op    = “add”;
            path  = “/fields/System.Title”;
            value = $title
}
) | ConvertTo-Json

$jsondata = $wc.UploadString($uri, “PATCH”, $data) | ConvertFrom-Json
$jsondata

}

The problem is I get into an authentication loop. It happens to be in Chrome, but you might find the same problem in other browsers.

It seems to be a recent issue, maybe related to MFA changes in AAD?

I used to be re-promoted for my ID when I swapped organisations in a browser tab, but not asked for further authentication

However, now the following happens

• I login to an organisation without a problem e.g https://dev.azure.com/someorg using ID, password and MFA
• In the same browser window, when I connect to another organisation e.g. https://dev.azure.com/someotherorg 
• I am asked to pick an account, then there is the MFA challenge, but then go back to the login
• …. and repeat.

The fix is to go in the browser tab to https://dev.azure.com. As you are already authenticated you will be able to sign out, then all is OK, you can login again.

The other options is to make even more use of Chrome People; one ‘person’ per customer, as opposed to my current usage on one ‘person’ per ID

As the service is a basic REST service and takes a bit of time to start-up I though a gate was a perfect way to pause the release pipeline until service was ready for the automated tests.

However, I hit a problem, the gates always failed as the internal server could not be resolved.

After a bit of thought I realised why. Gates as actual agentless tasks, they don’t run on the agent but on the server, so are outside the firewall. They could never connect to the private service without ports being opened, which was never going to happen.

So at this point in time I can’t use gates on this pipeline. Any similar logic to do the same job would have to be developed as scripts I can run on an agent.

Setup

Folders

Assume you have a Git repo with source for the UI, backend Services and common code in sub folders

/ [root]
     UI
     Services
     Common

Branch Policies

On the Master branch there are a policies of running

• one build for anything in the UI folder/project or common folder/project
• and a different build for anything in the Services folder/project or common folder/project

These build were filtered by path using the filters

/UX; /Common
/Services; /Common

The Issue

I discovered the problem by doing the following

• Create a PR for some work that effects the UI project
• As expected the UI build triggers
• Update the PR with a second commit for the Services code
• The Service build is not triggered

The Solution

The fix was simple it turns out. Remove the spaces from the filter paths so they become

/UX;/Common
/Services;/Common

Once this was done the builds triggered as expected.

Thanks again to the Azure DevOps Product Group for the help

I use Azure DevOps Pipelines for the build and release of my Azure DevOps Pipeline extensions, I previously detailed my process here .

For a good few months now YAML builds have been available. These provide the key advantage that the build is defined in a YAML text file that is stored with your product’s source code, thus allowing you to more easily track build changes. Also bulk editing becomes easier as a simple text editor can be used.

I have been putting off moving my current GUI based builds for as there is a bit of work, this post document then step.

Process

Getting the old build content

First I created a new branch in my local copy of my GitHub repo that stores the source for my extensions

I then created an empty file azure-pipelines-build.yaml the same folder as the root of the extension I was replacing the build for. I created the empty text file. I did this as the current create new build UI allows you to pick a file or a create one, but if you create one it gives you no control as to where or how it is named

In you existing build I then clicked the pipeline level ‘View YAML’

Note:  Initially I found this link disabled, but if you click around the UI, into the task details, variables etc, it eventually becomes enabled. I have no idea why.

Copy this YAML into you newly created azure-pipelines-build.yaml file, committed the file and pushed it GitHub as the new branch.

Creating the YAML build

I then created a new YAML based build, picking in my case GitHub as the source host, the correct branch, and correct file.

This YAML contains the  core of what is needed, but the build was missing some some items such as triggers, build number and variable.

I added

• the name (build number)
• the PR triggers to the YAML

to the .YAML file, but decided to declare my variables as they contained secrets within the build definition in Azure DevOps.

The final YAML file was can be viewed here

What I fixed in passing

In the past I used to package up my extensions twice, once packaged as private (for testing) and once as public. This was due to the limitation of the Azure DevOps Marketplace and the release tasks I was using at the time. Whilst passing a took the chance to change to only building the public VSIX package, but updated my release pipeline process to dynamically inject the settings for private testing. This was done using the newer Azure DevOps Extensions Tasks.

As I side note I had to upgrade to these newer release tasks anyway as the older ones had ceased to work due to using old API calls

Swapping in the new build into the release process

To replace the old GUI build with the new YAML build I did the following

• Renamed my old GUI build and disabled this (the disable is vital else it continues to be triggered by the GitHub PRs, even if the triggers are removed in the build)
• Renamed my new YAML build to the old GUI build name (not vital, but it felt neater)
• Updated my release pipeline to pick the new YAML build as opposed to the old GUI build. Even though the names were the same, their internal IDs are not, so this needs to be swapped. I made sure my ‘source alias’ did not change, so I did not have to make other changes to my release pipeline. 

Once this was done I triggered a new GitHub PR and everything worked as expects.

What Next

I have kept the old build about just in case there is a problem I have not spotted, but I intend to delete this soon.

I now need to make the same changes for all my other build. The only difference for from this process will be for builds that make use of Task Groups, such as all those for Node based extensions. Task Groups cannot be exported as YAML at this time, so I will have to manually rebuilding these steps in a text editor. So more prone to human error, but I think it needs to be done.

So a nice back burner project. I will probably update them as release new versions of extensions.

Microsoft have just released a VSCode extension to help address this problem – it is called Azure Pipelines

I have yet to give it a really good workout, but first impressions are good.

It does not remove the need for good documentation of task options, there is a need for my script to generate YAML documentation from a task.json file, but anything extra to ease editing helps.

If you are like me for historic reasons you have multiple Azure DevOps organisations (instances) backed by the same Azure Active Directory (AAD). In my case for example: one was created when Azure DevOps was first released as TFSPreview.com and another is from our migration from on-prem TFS using the DB Migration Tools method; and I have others. I make active use of all of these for different purposes, though one is primary with the majority of work done on it, and so I want to make sure the inherited process templates are the same on each of them. Using the primary organisation as the master customisation. Note I have already converted all my old on-premises XML process models to inherited process templates. There is no out the box way to do keep processes in syncs, but it is possible using a few tools. The main one is the Microsoft Process Migrator for Node on GitHub.

The Solution

Firstly I cloned the Microsoft Process Migrator and built it as per the instructions on the repo. I created a config file and then ran the tool. On one organisation it ran fine. However on another I had errors like: [ERROR] [2018-11-26T14:35:44.880Z] Process import validation failed. Process with same name already exists on target account. [ERROR] [2018-11-26T14:39:54.206Z] Import failed, see log file for details. Create field ‘Location’ failed, see log for details This was because I had in the past manually duplicated the inherited process template onto this organisation, so there was a process with the same name and fields of the same names. The first error was easy to fix, import the template with a new (temporary) name. The second is more problematic. I had two choice

• A manual fix
• An automated fix using Migration Tools for Azure DevOps from Martin Hinshelwood

As I only had a few duplicated unused fields on a single organisation I picked the former. If I had many organisations to sort out I would picked the latter. So my process ended up being

1. Run the Microsoft Process Migrator to migrate ‘My Process’ on the source organisation to ‘My Process 1’ on the target organisation
2. It gave an error, providing the name of the duplicated field
3. I checked on the target organisation using a work item query that the field was empty or only had defaulted data (if it had not been I would have used Martin’s tool to migrate the data to a temporary field and then deleted the problem field, moving the data back to the correct field from the temporary field when the import of the process template was completed)
4. I deleted the field from the work item type that referenced it
5. I deleted the field
6. I deleted the process template ‘My Process 1’, a failed import leaves a half created process
7. I went back to step 1 and repeated until the import completed without error
8. I tested my migrated inherited process was OK
9. On the target organisation I then renamed ‘My Process’ to ‘My Process – Old’
10. I then renamed ‘My Process 1’ to ‘My Process’
11. In my case I also made ‘My Process’ as the default, you might not do this if another process is the default, but step 13 does require the process template is not the default
12. I moved all the team projects using the process template now called ‘My Process – Old’ to ‘My Process’
13. I was then able to delete the process template ‘My Process – Old’ as it has no associated team projects and was not the default

As I customise my primary organisation’s process templates I can repeat this process to keep the processes in sync between organisations.  Note that in future migrations I won’t have to do steps 2..6 as there are no manually created duplicated fields. So it should be more straight forward. So a valid solution until any similar functionality is built into Azure DevOps, and there is no sign of that on the roadmap.

Did you know that there was a REST API alternative to TFS Aggregator?

My Azure DevOps Services & Server Alerts DSL is out there, and has been for a while, but I don’t think used by many people. It aims to do the same as TFS Aggregator, but is based around Python scripting.

However, I do have to say it is more limited in flexibility as it has only been developed for my (and a few of my clients needs), but its an alternative that is based on the REST APIs. 

Scripts are of the following form, this one changes the state of a work item if all it children are done

import sys
# Expect 2 args the event type and a value unique ID for the wi
if sys.argv\[0\] == "workitem.updated" : 
    wi = GetWorkItem(int(sys.argv\[1\]))
    parentwi = GetParentWorkItem(wi)
    if parentwi == None:
        LogInfoMessage("Work item '" + str(wi.id) + "' has no parent")
    else:
        LogInfoMessage("Work item '" + str(wi.id) + "' has parent '" + str(parentwi.id) + "'")

        results = \[c for c in GetChildWorkItems(parentwi) if c\["fields"\]\["System.State"\] != "Done"\]
        if  len(results) == 0 :
            LogInfoMessage("All child work items are 'Done'")
            parentwi\["fields"\]\["System.State"\] = "Done"
            UpdateWorkItem(parentwi)
            msg = "Work item '" + str(parentwi.id) + "' has been set as 'Done' as all its child work items are done"
            SendEmail("richard@blackmarble.co.uk","Work item '" + str(parentwi.id) + "' has been updated", msg)
            LogInfoMessage(msg)
        else:
            LogInfoMessage("Not all child work items are 'Done'")
else:
	LogErrorMessage("Was not expecting to get here")
	LogErrorMessage(sys.argv)

I have recently done a fairly major update to the project. The key changes are:

• Rename of project, repo, and namespaces to reflect Azure DevOps (the namespace change is a breaking change for existing users)

• The scripts that are run can now be

• A fixed file name for the web instance running the service

• Based on the event type sent to the service

• Use the subscription ID, thus allowing many scripts (new)

• A single instance of the web site running the events processor can now handle calls from many Azure DevOps instances.

• Improved installation process on Azure (well at least tried to make the documentation clearer and sort out a couple of MSDeploy issues)

Full details are on the project can be seen on the solutions WIKI, maybe you will find it of use. Let me know if the documentation is good enough

This has shown up a hole in my Pipeline Tasks documentation, I had nothing on YAML!

So I have added a YAML usage page for each set of tasks in each of my extensions e.g the file utilities tasks.

Now, as are most developers, I am lazy. I was not going to type all that information. So I wrote a script to generate the markdown from respective task.json files in the repo. Now this script will need some work for others to use as it relies on some special handling due to quirks of my directory structure, but I hope it will be of use to others.

A really good open and detailed root cause analysis has just been posted by the Azure DevOps team at Microsoft. It also covers the mitigations they are putting place to make sure this same issues do not occur again.

We all have to remember that the cloud is not magic. Cloud service providers will have problems like any on-premise services; but trying to hide them does nothing to build confidence. So I for one applaud posts like this. I just wish all cloud service providers were as open when problem occur.

I have considered splitting the GitHub repo, but as a number of people have forked it, over 100 at the last count, I did not want to start a chain of chaos for loads of people.

This initial choice has meant that until very recently I could not use the Pull Request triggers in Azure DevOps Pipelines against my GitHub repo. This was because all builds associated with the repo triggered on any extension PR. So, I had to trigger builds manually, providing the branch name by hand. A bit of a pain, and prone to error.

I am pleased to say that with the roll out of Sprint 140 we now get the option to add a path filter to PR triggers on builds linked to GitHub repo; something we have had for Azure DevOps hosted Git repos since Sprint 126.

So now my release process is improved. If I add a path filter as shown below, my build and hence release process trigger on a PR just as I need.

It is just a shame that the GitHub PR only checks the build, not the whole release, before saying all is OK. Hope we see linking to complete Azure DevOps Pipelines in the future.

The next major release of TFS will not be named TFS 2019 as you might have expected, but will use the new name of Azure DevOps Server. You can see the features planned for this next release in the Azure DevOps Features Timeline

The documentation for the VSTS WIKI suggests you can embed a video in a VSTS WIKI using the markdown/HTML```

Problem is that this does not seem to work, the MP4 just does not appear, you get an empty video player. However, if you swap to a full URL it does work e.g.

### The Workaround

The workaround is to either place the MP4 file in some URL accessible location e.g. some Azure web space (not really addressing the problem), or more usefully use the [VSTS API to get the file](https://docs.microsoft.com/en-us/rest/api/vsts/git/items/get?view=vsts-rest-4.1) out the repo that backs the WIKI. The format of the HTML tag becomes```
<video src="https://vstsinstance.visualstudio.com/MyTeamProject/\_apis/git/repositories/MyTeamProject.wiki/Items?path=\_media%2Fvstswiki\_high.mp4 width=400" controls>
</video>
```This will get the current version of the file on default branch, you can add extra parameters to to specify versions and branches if required as per the [API documentation](https://docs.microsoft.com/en-us/rest/api/vsts/git/?view=vsts-rest-4.1). So not a perfect solution as you have to think about branches and versions, they are not handled automatically, but at least it does work

As the user who’s PAT I was using was a Build Administrator I was a bit confused, but then I remembered to check their user access level. It was set to Stakeholder, once this was changed to Basic I was able to register the agent without use.

Also, so as to not use up a Basic license, when I did not need to, I swapped the user back to being a Stakeholder once the agent was registered. This can be done as the token used for the actual build is not the one used to register it, but one assigned at build time by VSTS.

If you used the TFS DB migration service you ended up with a VSTS instance full of the the XML style team projects, and you were stuck there, with no way to change these to the new inherited mode, that is until now as Microsoft have released to preview a conversion tool.

I have been trying this tool to migrate all our active XML based team projects to Inherited equivalents, and it has worked very well for me, but I have to say, we don’t have any hugely complex customisations, so your mileage may vary depending on how much you modified your XML based team project templates.

However, I wanted to go further than the basic process as documented

Since moving to VSTS all our new team project have been created  using an inherited template based on Scrum. I wanted to move all our active XML based team project to this standardised template. This took a little extra work.

The basic process is easy as to change Inherited Process you

1. In the instance admin page https://\[instance\].visualstudio.com/\_settings/process pick the target template
2. Click on the ellipse (…) and pick ‘Change team project to use ….’
3. Pick the team project you wish to migrate and you are done

REMEMBER: A really nice touch (with XML and Inherited templates) is that you can just switch back if you don’t like the result, no data is lost when you swap templates, but some of it might be hidden as fields are not shown on the new work item types.

The problem I had was that our old XML based team project templates had different customisation to our current inherited standard. To address this I needed to

• Adding a few fields to our standard inherited template based on Scrum, for critical legacy customisations
• In one case a work item types in use in the XML template had no match in our current template. In this case I changed the work item type to it’s equivalent (we had used a variety of ‘flavours of PBI’ so this was not a major problem), adding a tag to the work items to they could be identified, then deleting the offending work item type.
• I could conceive of a need for more complex remapping for work item types and fields, but I was able to avoid this.

So once done, I was then able to migrate all the team project to my target process template.

So a very nice experience, and one that now means we can make sure all our team projects use the same set of customisation. No longer do we need to worry about customising in XML and Inherited models.

In Team Explorer, when I came to do a ‘Map & Get’ to map the source locations I got a ‘TF14061: The workspace ws_1_18;Project Collection Build Service does not exist’ error

Strange error, which I could see no obvious reason for. Turns out the work around was just to press the ‘Advanced’ link/button and accept the defaults

I have a VSTS build extension that can apply a VSTS generated build number to Android APK packages. This takes a VSTS build number and generates, and applies, the Version Name (a string) and Version Code (an integer) to the APK file manifest.

The default parameters mean that the behaviour of this task is to assume (using a regular expression) the VSTS build number has at least three fields major.minor.patch e.g. 1.2.3, and uses the 1.2 as the Version Name and the 3 as the Version Code.

Now, it is important to note that the Version Code must be a integer between 1 and 2100000000 and for the Google Play Store it must be incrementing between versions.

So maybe these default parameter values for this task are not the best options?

The problem the way we use the task

When we use the Android Manifest Versioning task for our tuServ Android packages we use different parameter values, but we recently found these values still cause a problem.

Our VSTS build generates  build numbers with four parts $(Major).$(Minor).$(Year:yy)$(DayOfYear).$(rev:r)

• $(Major) – set as a VSTS variable e.g. 1
• $(Minor) – set as a VSTS variable e.g. 2
• $(Year:yy)$(DayOfYear) – the day for the year e.g. 18101
• $(rev:r) – the build count for the build definition for the day e.g. 1

So we end up with build numbers in the form 1.2.18101.1

The Android version task is set in the build to make

• the Version Number {1}.{2}.{3}.{4}  - 1.2.18101.1
• the Version Code {1}{2}{3}{4} – 12181011

The problem is if we do more than 9 builds in a day, which is likely due to our continuous integration process, and release one of the later builds to the Google Play store, then the next day any build with a lower revision than 9 cannot be released to the store as its Version Code is lower than the previously published one e.g.

• day 1 the published build is 1.2.18101.11 so the Version Code is 121810111
• day 2 the published build is 1.2.18102.1 so the Version Code is 12181021

So the second Version Code is 10x smaller, hence the package cannot be published.

The Solution

The answer in the end was straightforward and found by one of our engineers Peter (@sarkimedes). It was to change the final block of the VSTS build number to $(rev:rrr), as detailed in the VSTS documentation. Thus zero padding the revision from .1 to .001. This allows up to 1000 builds per day before the problem of altering the Version Code order of magnitude problem occurs. Obviously, if you think you might do more than 1000 internal builds in a day you could zero pack as many digits as you want.

So using the new build version number

• day 1 the published build is 1.2.18101.011 so the Version Code is 1218101011
• day 2 the published build is 1.2.18102.001 so the Version Code is 1218102001

So a nice fix without any need to alter the Android Manifest Versioning task’s code. However, changing the default Version Code parameter to {1}{2}{3} is probably advisable.

A minor issue is that this re-write has introduced a couple of breaking changes, as detailed below and on the project wiki

• oAuth script access has to be enabled on the agent running the task

• There are minor changes in the template format, but for the good, as it means both TFVC and GIT based releases now use a common template format. Samples can be found in the project repo

Because of the breaking changes, we made the decision to release both V1 and V2 of the task in the same extension package, so not forcing anyone to update unless they wish to. A technique I have not tried before, but seems to work well in testing.

Hope people still find the task of use and thanks again to Greg for all the work on the extension

Azure is a highly resilient service, and VSTS has excellent SLAs. However, a question that is often asked is ‘How do I backup my VSTS instance?’. The simple answer is you don’t. Microsoft handle keeping the instance up, patched and serviceable. Hence, there is no built in means for you to get a local copy of all your source code, work items or CI/CD definitions. Though there have been requests for such a service. This can be an issue for some organisations, particularly for source control, where there can be a need to have a way to keep a private copy of source code for escrow, DR or similar purposes.

A Solution

To address this issue I decided to write a PowerShell script to download all the GIT and TFVC source in a VSTS instance. The following tactics were used

• Using the REST API to download each project’s TFVC code as a ZIP file. The use of a ZIP file avoids any long file path issues, a common problem with larger Visual Studio solutions with complex names
• Clone each Git repo. I could download single Git branches as ZIP files via the API as per TFVC, but this seemed a poorer solution given how important branches are in Git.

So that the process was run on regular basis I designed it to be run within a VSTS build. Again here I had choices:

• To pass in a Personal Access Token (PAT) to provide access rights to read the required source to be backed up. This has the advantage the script can be run inside or outside of a VSTS build. It also means that a single VSTS build can backup other VSTS instances as long as it has a suitable PAT for access
• To use the System Token already available to the build agent. This makes the script very neat, and PATs won’t expire, but means it only works within a VSTS build, and can only backup the VSTS instance the build is running on.

I chose the former, so a single scheduled build could backup all my VSTS instances by running the script a number of time with different parameters

To use this script you just pass in

• The name of the instance to backup

• A valid PAT for the named instance

• The path to backup too, which can be a UNC share assuming the build agent has rights to the location

What’s Next

The obvious next step is to convert the PowerShell script to be a VSTS Extension, at this point it would make sense to make it optional to use a provided PAT or the System Access Token. Also I could add code to allow a number of multiple cycling backups to be taken e.g. keep the last 3 backups These are maybe something for the future, but they don’t really seems a good return in investment at this time to package up a working script as an extensions just for a single VSTS instance.

Note, in my CI/CD process I have a private and public version of each extension (set of tasks), the former is used for functional testing within the CD process, the latter is the one everyone can see.

So, this meant I had two public versions of the same extension, confusing.

Turns out you can’t change a public extension back to be private, either via the UI or by uploading a corrected VSIX. Also you can’t delete any public extension that has ever been downloaded, and my previously private one had been downloaded once, by me for testing.

So my only option was to un-publish the previously private extension so only the correct version was visible in the public marketplace.

This meant I had to also alter my CI/CD process to change the extensionID of my private extension so I could publish a new private version of the extension.

Luckily, as all the GUIDs for the tasks within the extension did not change once I had installed the new version of the extension I had mispublished in my test VSTS instance my pipeline still worked.

Only downside is I am left with an un-publish ‘dead’ version listed in my private view of the marketplace. This is not a problem, just does not look ‘neat and tidy’

What did not work

My first thought was to use the Invoke HTTP REST API gate, calling the VSTS API https://.visualstudio.com/_apis/distributedtask/tasks/. This API call returns a block of JSON containing details about the deployed task visible to the specified VSTS instance. In theory you can parse this data with a JSONPATH query in the gates success criteria parameter to make sure the correct version of the task is deployed e.g. eq($.value[?(@.name == “BuildRetensionTask”)].contributionVersion, “1.2.3”) However, there is a problem. At this time the Invoke HTTP REST API gate task does not support the == equality operator in it’s success criteria field. I understand this will be addressed in the future, but the fact it is currently missing is a block to my current needs. Next I thought I could write a custom VSTS gate. These are basically ‘run on server’ tasks with a suitably crafted JSON manifest. The problem here is that this type of task does not allow any code (Node.JS or PowerShell) to be run. They only have a limited capability to invoke HTTP APIs or write messages to service bus. So I could not implement the code I needed to process the API response. So another dead end.

What did work

The answer, after a suggestion from the VSTS Release Management team at Microsoft, was to try the Azure Function gate. To do this I created a new Azure Function. I did this using the Azure Portal, picking the consumption billing model, C# and securing the function with a function key, basically the default options. I then added the C# function code (stored in GitHub), to my newly created Azure Function. This function code takes

• The name of the VSTS instance
• A personal access token (PAT) to access the VSTS instance
• The GUID of the task to check for
• And the version to check for

It then returns a JSON block with true or false based on whether the required task version can be found. If any of the parameters are invalid an API error is returned By passing in this set of arguments my idea was that a single Azure Function could be used to check for the deployment of all my tasks. Note: Now I do realise I could also create a release pipeline for the Azure Function, but I chose to just create it via the Azure Portal. I know this is not best practice, but this was just a proof of concept. As usual the danger here is that this proof of concept might be one of those that is too useful and lives forever!

To use the Azure Function

Using the Azure function is simple

• • Added an Azure Function gate to a VSTS release
  • Set the URL parameter for the Azure Function. This value can be found from the Azure Portal. Note that you don’t need the Function Code query parameter in the URL as this is provided with the next gate parameter. I chose to use a variable group variable for this parameter so it was easy to reuse between many CD pipelines
  • Set the Function Key parameter for the Azure Function, again you get this from the Azure Portal. This time I used a secure variable group variable
  • Set the Method parameter to POST
  • Set the Header content type as JSON

{
      "Content-Type": "application/json"
}

• • Set the Body to contain the details of the VSTS instance and Task to check. This time I used a mixture of variable group variables, release specific variables (the GUID) and environment build/release variables. The key here is I got the version from the primary release artifact $(BUILD.BUILDNUMBER) so the correct version of the tasks is tested for automatically

{
     "instance": "$(instance)",
     "pat": "$(pat)",
     "taskguid": "$(taskGuid)",
     "version": "$(BUILD.BUILDNUMBER)"
}

• Finally set  the Advanced/Completion Event to ApiResponse with the success criteria of``` eq(root[‘Deployed’], ’true’)

Once this was done I was able to use the Azure function as a VSTS gate as required

Summary

So I now have a gate that makes sure that for a given VSTS instance a task of a given version has been deployed. If you need this functionality all you need to do is create your own Azure Function instance, drop in my code and configure the VSTS gate appropriately. When equality == operator becomes available for JSONPATH in the REST API Gate I might consider a swap back to a basic REST call, it is less complex to setup, but we shall see. The Azure function model does appear to work well

Having automated builds is essential to any good development process. Irrespective of the build engine in use, VSTS, Jenkins etc. you need to have a means to create the VMs that are running the builds.

You can of course do this by hand, but in many ways you are just extending the old ‘it works on my PC – the developer can build it only on their own PC’ problem i.e. it is hard to be sure what version of tools are in use. This is made worse by the fact it is too tempting for someone to remote onto the build VM to update some SDK or tool without anyone else’s knowledge.

In an endeavour to address this problem we need a means to create our build VMs in a consistent standardised manner i.e a configuration as code model.

At Black Marble we have been using Lability to build our lab environments and there is no reason we could not use the same system to create our VSTS build agent VMs

• Creating base VHDs disk images with patched copies of Windows installed (which we update on a regular basis)
• Use Lability to provision all the required tools – this would need to include all the associated reboots these installers would require. Noting that rebooting and restarting at the correct place, for non DSC based resources, is not Lability’s strongest feature i.e. you have to do all the work in custom code

However, there is an alternative. Microsoft have made their Packer based method of creating VSTS Azure hosted agents available on GitHub. Hence, it made sense to me to base our build agent creation system on this standardised image; thus allowing easier migration of builds between private and hosted build agent pools whether in the cloud or on premises, due to the fact they had the same tools installed.

The Basic Process

To enable this way of working I forked the Microsoft repo and modified the Packer JSON configuration file to build Hyper-V based images as opposed to Azure ones. I aimed to make as few changes as possible to ease the process of keeping my forked repo in sync with future changes to the Microsoft standard build agent. In effect replacing the builder section of the packer configuration and leaving the providers unaltered

So, in doing this I learnt a few things

Which ISO to use?

Make sure you use a current Operating System ISO. First it save time as it is already patched; but more importantly the provider scripts in the Microsoft configuration assume certain Windows features are available for installation (Containers with Docker support specifically) that were not present on the 2016 RTM ISO

Building an Answer.ISO

In the sample I found for the Packer hyperv-iso builder the AutoUnattended.XML answers file is provided on an ISO (as opposed to a virtual floppy as floppies are not support on Gen2 HyperV VMs). This means when you edit the answers file you need to rebuild the ISO prior to running Packer.

The sample script to do this has lines to ‘Enable UEFI and disable Non EUFI’; I found that if these lines of PowerShell were run the answers file was ignored on the ISO. I had to comment them out. It seems an AutoUnattended.XML answers file edited in VSCode is the correct encoding by default.

I also found that if I ran the PowerShell script to create the ISO from within VSCode’s integrated terminal the ISO builder mkisofs.exe failed with an internal error. However, it worked fine from a default PowerShell windows.

Installing the .NET 3.5 Feature

When a provider tried to install the .NET 3.5 feature using the command

Install-WindowsFeature -Name NET-Framework-Features -IncludeAllSubFeature

it failed.

Seems this is a bug in Windows 2016 and the workaround is to specify the –Source location on the install media

Install-WindowsFeature -Name NET-Framework-Features -IncludeAllSubFeature -Source “D:sourcessxs”

Once the script was modified in this manner it ran without error

Well how long does it take?

The Packer process is slow, Microsoft say for an Azure VM it can take up to over 8 hours. A HyperV VM is no faster.

I also found the process a bit brittle. I had to restart the process a good few times as….

• I ran out of disk space (no unsurprising this broke the process)
• The new VM did not get a DHCP assigned IP address when connected to the network via the HyperV Default Switch. A reboot of my HyperV host PC fixed this.
• Packer decided the VM had rebooted when it had not – usually due to a slow install of some feature or network issues
• My Laptop went to sleep and caused one of the above problems

So I have a SysPrep’d VHD now what do I do with it now?

At this point I have options of what to do with this new exported HyperV image. I could manually create build agent VM instances.

However, it appeals to me to use this new VHD as a based image for Lability, replacing our default ‘empty patched Operating System’ image creation system, so I have a nice consistent way to provision VMs onto our Hyper-V servers.

• Add the GitVersion task at the start of the build, with its default parameters
• Add my JSON versioning task with default parameters apart from
  • Version Number set to $(GitVersion.SemVer)
  • Use Version Number without Processing (Advanced) checked
  • Filename Pattern (Advanced) set to azuredeploy.json
  • Field to update (Advanced) set to contentVersion

In the logs you see output similar to the following``` Source Directory: E:Build2_work361s Filename Pattern: azuredeploy.json Version Number/Build Number: 0.1.0-unstable.843 Use Build Number Directly: true Version Filter to extract build number: d+.d+.d+.d+ Version Format for JSON File: {1}.{2}.{3} Field to update (all if empty): contentVersion Output: Version Number Parameter Name: OutputedVersion Using the provided build number without any further processing JSON Version Name will be: 0.1.0-unstable.843 Will apply 0.1.0-unstable.843 to 12 files. Updating the field ‘contentVersion’ version Existing Tag: contentVersion": “1.0.0.0” Replacement Tag: contentVersion": “0.1.0-unstable.843” …

For this PowerShell code to work you do need make some security changes to allow the build agent service user to write to the Git repo. This is documented by Microsoft. The PowerShell task to run this code is placed in a build as the only task This build is then triggered as part of the release process Note that the triggering of this build has to be such that it runs on a non-blocking build agent as discussed in my previous posts. In my case I trigger the build to add the extra commits and work items just before triggering the validation build on my private Azure hosted agent. Now, there is no reason you can’t just run the PowerShell directly within the release if you wanted to. I chose to use a build so that the build could be reused between different VSTS extension CI/CD pipelines; remember I have two Generate Release Note Extensions, PowerShell and NodeJS Based. So another step to fully automating the whole release process.

1.  They get a list of files
2.  Extract a version number from the build number
3.  Then apply this to one or more files in a product/task specific manner

there has been some cut and paste coding. This means that I have NPM modules in the tasks **package.json** file that were not needed for a given task. I could manually address this but there is an NPM module to help, DepCheck. First install the [DepCheck](https://www.npmjs.com/package/depcheck) module```
npm install depcheck –g
```then run **depcheck** from the command line whist within your task’s folder. This returns a list of modules listed in the **package.json** that are not referenced in the code files. These can then be removed from the **package.json.**  e.g. I saw```
Unused dependencies
\* @types/node
\* @types/q
\* Buffer
\* fs
\* request
\* tsd
Unused devDependencies
\* @types/chai
\* @types/mocha
\* @types/node
\* mocha-junit-reporter
\* ts-loader
\* ts-node
\* typings
```The important ones to focus on are the first block (non-development references), as these are the ones that are packaged with the production code in the VSIX; I was already pruning the **node\_module** folder of development dependencies prior to creating the VSIX to remove devDependancies using the command```
npm prune –production
```I did find some of the listed modules strange, as I knew they really were needed and a quick test of removing them did show the code failed if they were missing. These are what [depchecks documentation calls false alerts](https://www.npmjs.com/package/depcheck). I found I could remove the **@type/xxx** and **tsd** references, which were the big ones, that are only needed in development when working in TypeScript. Once these were removed for all four of my NodeJS based tasks my VSIX dropped in size from 22Mb to 7Mb. So problem solved.

The first is for adding a version to a file like a package.json file e.g.

{  
"name": "myapp",  
"version": "1.0.0",  
"license": "MIT"  
}

The second is designed for angular environment.ts file e.g.

export const environment = {  
production: true,  
version: '1.0.0.0'  
};

But I bet people find other uses, they always do.

You can find the extension in the marketplace, you need 1.31.x or later to see the new versioner tasks

I have recently been posting on using Azure to host private VSTS build/release agents to avoid agent queue deadlocking issues with more complex release pipelines. One of the areas discussed is reducing cost of running a private agent in Azure by only running the private agent within a limited time range, when you guess it might be needed. I have done this using DevTest Labs Auto Start and Auto Stop features. This works, but is it not better to only start the agent VM when it is actually really needed, not when you guess it might be? I need this private agent only when working on my VSTS extensions, not something I do everyday. Why waste CPU cycles that are never used?

New VSTS Extension

I had expected there would already be a VSTS  extension to Start and Stop DevTest Lab VMs, but the Microsoft provided extension for DevTest Labs only provides tasks for the creation and deletion of VMs within a lab. So I am pleased to announce the release of my new DevTest Labs VSTS Extension to fill this gap, adding tasks to start and stop a DevTest Lab VM on demand from within a build or a release. My Usage I have been able to use the tasks in this extension to start my private Azure hosted agent only when I need it for functional tests within a release. However, they could equally be used for a variety of different testing scenarios where any form of pre-built/configured VMs needs to be started or stopped as opposed to slower processes of creating/deploying a new deployment of a DevTest lab VM. In may case I added an extra agent phases to my release pipeline to start the VM prior to it being needed. I could also have used another agent phase to stop the VM once the tests were completed. However, I made the call to leave the VM running and let DevTest Labs’ Auto Stop shut it down at the end of the day. The reason for this is that VM start up and shutdown is still fairly slow, a minute or two, and I often find I need to run a set of function tests a few times during my development cycle; so it is a bit more efficient to leave the VM running until the end of the day. Only taking the start-up cost once. You may have course have different needs, hence providing both the Start and Stop Tasks

Development

This new extension aims to act as a supplement to the Microsoft provided Azure DevTest Lab Extension. Hence to make development and adoption easier, it uses exactly the same source code structure and task parameters as the Microsoft provided extension. The task parameters being:

• Azure RM Subscription - Azure Resource Manager subscription to configure before running.
• Source Lab VM ID - Resource ID of the source lab VM. The source lab VM must be in the selected lab, as the custom image will be created using its VHD file. You can use any variable such as $(labVMId), the output of calling Create Azure DevTest Labs VM, that contains a value in the form /subscriptions/{subId}/resourceGroups/{rgName}/providers/Microsoft.DevTestLab/labs/{labName}/virtualMachines/{vmName}.

The issue I had was that the DevTest Labs PowerShell API did not provide a command to start or stop a VM in a lab. I needed to load the Azure PowerShell library to use the Invoke-AzureRmResourceAction  command. This requires you first call Login-AzureRmAccount to authenticate prior to calling the actual Invoke-AzureRmResourceAction required. This required a bit of extra code to get and reuse the AzureRM endpoint to find the authentication details.``` # Get the parameters $ConnectedServiceName = Get-VstsInput -Name “ConnectedServiceName”

Get the end point from the name passed as a parameter

$Endpoint = Get-VstsEndpoint -Name $ConnectedServiceName -Require

Get the authentication details

$clientID = $Endpoint.Auth.parameters.serviceprincipalid $key = $Endpoint.Auth.parameters.serviceprincipalkey $tenantId = $Endpoint.Auth.parameters.tenantid $SecurePassword = $key | ConvertTo-SecureString -AsPlainText -Force $cred = new-object -typename System.Management.Automation.PSCredential -argumentlist $clientID, $SecurePassword

Authenticate

Login-AzureRmAccount -Credential $cred -TenantId $tenantId -ServicePrincipal Important to note that for this code to work you have to set the task’s **task.json** to run **PowerShell3** and package the Powershell VSTS API module in with the task. “execution”: { “PowerShell3”: { “target”: “$(currentDirectory)\StartVM.ps1”, “argumentFormat”: “”, “workingDirectory”: “$(currentDirectory)”     }   }

### In Summary

I have certainly found this extension useful, and I have leant more that I had expect I would about VSTS endpoints and Azure authentication. Hope it is useful to you too.

• 28th Nov 2017: Also see this second post Creating a VSTS build agent on an Azure DevLabs Windows Server VM with no GUI - Using Artifacts
• 30th Nov 2017: Also see associated post Announcing a new VSTS Extension for Starting and Stopping Azure DevTest Labs VMs

As I posted recently I have been trying to add more functional tests to the VSTS based release CI/CD pipeline for my VSTS Extensions, and as I noted depending on how you want to run your tests e.g. trigger sub-builds, you can end up with scheduling deadlocks where a single build agent is scheduling the release and trying to run a new build. The answer is to use a second build agent in a different agent pool e.g. if the release is running on the Hosted build agent use a private build agent for the sub-build, or of course just pay for more hosted build instances. The problem with a private build agent is where to run it. As my extensions are a personal project I don’t have a corporate Hyper-V server to run any extra private agents on, as I would have for an company projects. My MVP MSDN Azure benefits are the obvious answer, but I want any agents to be cheap to run, so I don’t burn through all my MSDN credits for a single build agent. To this end I created a Windows Server 2016 VM in DevLabs (I prefer to create my VMs in DevLabs as it makes it easier tidying up of my Azure account) using an A0 sizing VM. This is tiny so cheap; I don’t intend to ever do a build on this agent, just schedule releases, so need to install few if any tools, so the size should not be an issue. To further reduce costs I used the auto start and stop features on the VM so it is only running during the hours I might be working. So I get an admittedly slow and limited private build agent but for less that $10 a month. As the VM is small it makes sense to not run a GUI. This means when you RDP to the new VM you just get a command prompt. So how do you get the agent onto the VM and setup? You can’t just open a browser to VSTS or cut and paste a file via RDP, and I wanted to avoid the complexity of having to open up PowerShell remoting on the VM. The process I used was as follows:

1. In VSTS I created a new Agent Pool for my Azure hosted build agents
2. In the Azure portal, DevLabs I created a new Windows Server 2016 (1709) VM
3. I then RDP’d to my new Azure VM, in the open Command Prompt I ran PowerShell powershell
4. As I was in my users home directory, I  cd’d into the downloads folder cd downloads
5. I then ran the following PowerShell command to download the agent (you can get the current URI for the agent from your VSTS Agent Pool ‘Download Agent’ feature, but an old version will do as it will auto update. invoke-webrequest -UseBasicParsing -uri https://github.com/Microsoft/vsts-agent/releases/download/v2.124.0/vsts-agent-win7-x64-2.124.0.zip -OutFile vsts-agent-win7-x64-2.124.0.zip
6. You can then follow the standard agent setup instructions from the VSTS Agent Pool ‘Download Agent’ feature mkdir agent ; cd agent PS Add-Type -AssemblyName System.IO.Compression.FileSystem ; [System.IO.Compression.ZipFile]::ExtractToDirectory("$HOMEDownloadsvsts-agent-win7-x64-2.124.0.zip", “$PWD”)
7. I then configured the agent to run as a service, I exited back to the command prompt to do this this, so the commands were exit config.cmd

I now had an other build agent pool to use in my CI/CD pipelines at a reasonable cost, and the performance was not too bad either.

Click for a PDF version  

It mentions a few 3rd party tools in the flowchart, so here are some useful links

• TFS Integration Platform – can in theory move source and work items – but really try not to use it!
• VSTS Sync Migration Tools –  moves Work items
• Timely Migration – moves TFVC source (commercial product)
• OpsHub – moves Team Projects (free & commercial versions)
• Git TFS – move TFVC into Git
• TFS Office Integration – moves work items via Excel
• TFS Database Import Service – the full fidelity service

Also, if you find yourself in the orange box at the bottom and don’t want to use the TFS Database Import Service for some reason, have a look at this post I did on Microsoft’s UK Developers site. It might give you some ideas

  I have been developing a few VSTS/TFS build related extensions and have published a few in the VSTS marketplace. This has all been a somewhat manual process, a mixture of Gulp and PowerShell has helped a bit, but I decided it was time to try to do a more formal approach. To do this I have used Jesse Houwing’s VSTS Extension Tasks. Even with this set of tasks I am not sure what I have is ‘best practice’, but it does work. The doubt is due to the way the marketplace handles revisions and preview flags. What I have works for me, but ‘your mileage may differ’

My Workflow

The core of my workflow is that I am building the VSIX package twice, once as a private package and the other as a public one. They both contain the same code and have the same version number, they differ in only visibility flags I am not using a the preview flag options at all. I have found they do not really help me. My workflow is to build the private package, upload it and test it by sharing it with a test VSTS instance. if all is good publish the matched public package on the marketplace. In this model there is no need to use a preview, it just adds complexity I don’t need. This may not be true for everyone.

Build

The build’s job is to take the code, set the version number and package it into multiple VSIX package.

1. First I have the vNext build get my source from my GitHub repo.
2. I add two build variables $(Major) and $(Minor) that I use to manually manage my version number
3. I set my build number format to $(Major).$(Minor).$(rev:r), so the final .number is incremented until I choose to increment the major or minor version.
4. I then use one of Jesse’s tasks to package the extension multiple times using the extension tag model parameter. Each different package step uses different Visibility settings (circled in red). I also set the version, using the override options, to the $(Build.BuildNumber) (circled in green)
5. **[Updated Aug 2016] **Set the PublisherID and ExtensionID on the tasks, using a pair of build variables is a good idea here to avoid entering strings twice. It is important thay the PublisherID is entered with the correct case - it is case sensitive within the marketplace. Strange things happend of the PublisherID in a VSIX package differ from the one registered on the marketplace
6. As I am using the VSTS hosted build agent I also need to make sure I check the install Tfx-cli in the global setting section
7. I then add a second identical publish task, but this time there is no tag set and the visibility is set to public.
8. Finally I use a ‘publish build artifacts’ task to copy the VSIX packages to a drop location

Release

So now I have multiple VSIX packages I can use the same family of tasks to create a release pipeline. I create a new release linked to be a Continuous Deployment of the previously created build and set its release name format to Release-$(Build.BuildNumber) My first environment uses three tasks, all using the option - to work from a VSIX package. Note In all cases I am using the VSIX path in the format $(System.DefaultWorkingDirectory)/GenerateReleaseNotes.Master/vsix/--$(Build.BuildNumber).vsix. I am including the build number variable in the path as I chose to put all the packages in a single folder, so path wildcards are not an option as the task would not know which package to use unless I alter my build to put one VSIX package per folder. My tasks for the first environment are

1. Publish VSTS Extension – using my private package so it is added as a private package to the marketplace
2. Share VSTS Extension – to my test VSTS account
3. Install VSTS Extension – to my test VSTS account

For details in the usage of these tasks and setting up the link to the VSTS Marketplace see Jesse’s wiki If I only intend a extension to ever be private this is enough. However I want to make mine public so I add a second environment that has manual pre-approval (so I have to confirm the public release) This environment only needs single task

1. Publish VSTS Extension – using my public package so it is added as a public package to the marketplace

I can of course add other tasks to this environment maybe send a Tweet or email to publicise the new version’s release

Summary

So now I have a formal way to release my extensions. The dual packaging model means I can publish two different versions at the same time one privately and the other public It is now just a case of moving all my extensions over to the new model. Though I am still interested to hear what other people view are? Does this seem a reasonable process flow?

• Manual install from GitHub
• Install via Chocolaty
• Install via Nuget

If you own the build agent VM then any of these options are good, you can even write the NuGet restore into your build process itself. However there is a problem, both the first two options need administrative access as they put the Pester module in the $PSModules folder (under ‘Program Files’); so these can’t be used on VSTS’s hosted build system, where your are not an administrator So this means you are left with copying the module (and associated functions folder) to some local working folder and running it manually; but do you really want to have to store the Pester module in your source repo? My solution was to write a vNext build tasks to deploy the Pester files and run the Pester tests. The task takes two parameters

• The root folder to look for test scripts with the naming convention  *.tests.ps1. Defaults to $(Build.SourcesDirectory)*
• The results file name, defaults to $(Build.SourcesDirectory)Test-Pester.XML

The Pester task does not in itself upload the test results, it just throws and error if tests fails. It relies on the standard test results upload task. Add this task and set

• it to look for nUnit format files
• it already defaults to the correct file name pattern.
• IMPORTANT: As the Pester task will stop the build on an error you need to set the ‘Always run’ to make sure the results are published.

Once all this is added to your build you can see your Pester test results in the build summary You can find the task in my vNextBuild repo

1. Clone the repo
2. Build the tasks using Gulp
3. Upload the task you require to your VSTS or TFS instance

Once this is done you can add the task to your build. You probably won’t need to set any parameters as long as you have settings.stylecop files to define your StyleCop ruleset in the same folders as your .CSPROJ files (or are happy default rulesets). If you do want to set parameters your options are:

• TreatStyleCopViolationsErrorsAsWarnings - Treat StyleCop violations errors as warnings, if set to False any StyleCop violations will cause the build to fail (default false).

And on the advanced panel

• MaximumViolationCount - Maximum violations before analysis stops (default 1000)
• ShowOutput - Sets the flag so StyleCop scanner outputs progress to the console (default false)
• CacheResults - Cache analysis results for reuse (default false)
• ForceFullAnalysis - Force complete re-analysis (default true)
• AdditionalAddInPath - Path to any custom rule sets folder, the directory cannot be a sub directory of current directory at runtime as this is automatically scanned. This folder must contain your custom DLL and the Stylecop.dll and Stylecop.csharp.cs else you will get load errors
• SettingsFile - Path to single settings files to use for all analysis (as opposed to settings.stylecop files in project folders)

    When you run the build with the new task you should expect to see a summary of the StyleCop run on the right