More on when Azure DevOps variables are available in pipeline runs

Introduction I have previously blogged a good deal on Azure DevOps variable evaluation, see here and here, but the saga continues… When variables exist Today I realised, something I guess should have been obvious, that when you manually queue a run, pre-defined variables such as $(Build.SourceBranchName) are not available until the pipeline is compiled and starts running. This is because though there is a value in UI branch combo but this value is not in $(Build....

July 11, 2024 · 4 min · Richard Fennell

Checking out Git submodules when Azure DevOps Protected Access to repos is enabled

The Issue Whilst working on an Azure DevOps YAML pipeline for a solution that used Git Submodules we hit a problem with the checkout of the repo and submodule using the YAML jobs: - job: 'Build' steps: - checkout: self submodules: true It got the main Git repo, but failed with the following error git submodule sync git --config-env=http.https://myorg@dev.azure.com.extraheader=env_var_http.https://myorg@dev.azure.com.extraheader submodule update --init --force Submodule 'Library' (https://myorg@dev.azure.com/myorg/myproject/_git/Library) registered for path 'Library' Cloning into 'D:/a/1/s/Library'....

July 10, 2024 · 2 min · Richard Fennell

So my Azure DevOps TF30063 error was down to DNS again

The Issue I recently upgraded a clients Azure DevOps server from 2019 to 2022. This required a new application tier VM due to the change in supported versions of Windows Server between the two versions. Unfortunately, the client’s developers had always accessed the old Azure DevOps Server using the VMs FQDN, as opposed to a DNS managed alias. Hence, given they wanted to minimise change, the plan was to create a DNS Alias so they could continue to use the same URLs and TFVC workspaces mappings....

July 9, 2024 · 2 min · Richard Fennell

Where has the staging URL PR comment generated by my GitHub Actions workflow gone?

The Issue Last week I noticed that the staging URL that is normally output as a comment was missing from new GitHub PRs. Previously, this URL was added automatically by the Azure/static-web-apps-deploy GitHub Action for PRs in our Hugo based websites. After a bit of digging, I noticed a warning message in the logs of the Action that said: … Done Zipping App Artifacts Uploading build artifacts. Finished Upload. Polling on deployment....

July 2, 2024 · 2 min · Richard Fennell

Don't call your Azure DevOps YAML Deployment stage 'deployment' - strange things happen

Just a little reminder, probably to my future self, to not call your Azure DevOps YAML Deployment stage ‘deployment’. If you forget you can expect to waste plenty of time, like we did last week, with environment agents not picking up queued jobs with no diagnostic logging messages to give you a clue as to what is going on. This issue has been reported on StackOverflow where it was pointed out that the official documentation used ‘deployment’ as the stage name....

July 1, 2024 · 1 min · Richard Fennell

It's the SonarQube Elasticsearch indexes again

Background I just upgraded our Azure/Docker container hosted SonarQube instance from 10.5.1 to 10.6.0. This was partly due to our usual upgrade process, we try to upgrade within a week or so of a new release, but also to address a specific Java issue we started to see when using the SonarQube@6 Azure DevOps tasks. The error was the SonarQube analysis completed successfully, but the clean up process failed with this JVM error....

June 28, 2024 · 2 min · Richard Fennell

Passing dynamically sized object parameters to Azure DevOps Pipeline templates

The Problem I have an Azure DevOps YAML template that does some deployment actions using PowerShell. This template is used multiple locations. The problem is that the PowerShell step within the template needs a variable number of environment variables, set from values stored in an Azure DevOps variable Group. The number of variables is not fixed because they depend on the underlying commands the PowerShell script is triggering to do the deployment....

June 24, 2024 · 2 min · Richard Fennell

Why are my Azure DevOps Pipeline cache hits missing

I have blogged in the past about Caching NVD Vulnerability Dependency data on hosted Azure DevOps Pipeline agents. Using the cache is a great way to speed up slow builds. However, today I was surprised to find I was getting cache misses on my pipeline, even though I was sure the cache should have been hit. There are rules over how the cache is used: The cache is specific to a pipeline definition, so there is no sharing of the cache between pipeline definitions The cache is only created if the pipeline is successful (running the post run tasks) The cache only lasts 7 days but what I had not realised was the cache is also specific to the branch in a not so obvious way....

June 5, 2024 · 2 min · Richard Fennell

Still time to register for the Global DevOps Experience

There is still time to register for the free Global DevOps Experience which is being run on the 15th of June at many venues around the world, including Black Marble’s offices. The Global DevOps Experience is a day full of learning and fun, as we immerse you in the world of DevOps and AI. You will learn about the latest trends and technologies, and work in a team to solve challenging exercises based around a realistic business scenario....

June 3, 2024 · 1 min · Richard Fennell

Azure DevOps pipeline jobs failing to start

The Issue Whilst migrating some Azure DevOps classic pipelines to templated multi-stage YAML, I hit a problem that a job running on a self-hosted agent would not start. The YAML stage, which required approval, would be queued and approved, but the agent would just sit there with the message Starting job but would never start. The strange thing was even though the job was not started, the pipeline instantly showed as failed with no error message....

May 29, 2024 · 1 min · Richard Fennell