Tech Tips

I have blogged in the past about Caching NVD Vulnerability Dependency data on hosted Azure DevOps Pipeline agents. Using the cache is a great way to speed up slow builds. However, today I was surprised to find I was getting cache misses on my pipeline, even though I was sure the cache should have been hit. There are rules over how the cache is used: The cache is specific to a pipeline definition, so there is no sharing of the cache between pipeline definitions The cache is only created if the pipeline is successful (running the post run tasks) The cache only lasts 7 days but what I had not realised was the cache is also specific to the branch in a not so obvious way. My pipeline was triggered off a PR, so the cache was being created on the ‘branch’ PR #123. This was working as expected, all runs of the PR triggered build used the cache after the initial run. However, if I manually triggered pipeline run of the same branch as the PR was using, there was a cache miss. ...

Background Programmatic connection to Azure DevOps cannot be done with your Active Directory credentials. This is because this involves a dialog being shown, and these days usually an MFA check too. Historically, the solution to this problem was to enable Alternate Credentials, which could be passed as username and password, without the dialog being shown. However, the use of these has been deprecated since 2020, and they have been completely removed since Jan 2024. ...

It is a major effort, often unfortunately ignored, keeping the dependencies in an open source project up to date. This was highlighted in Jesse Houwing’s post on the state of the Azure DevOps Marketplace. Since reading this I have made much more of an effort to keep my Azure DevOps Extensions up to date. Dependabot generated PRs have been a great help in this regard. Creating PRs for vulnerabilities and out of date dependencies. ...

The Issue We saw an issue with our SonarQube 10.3 Developer Edition (that is running as a Docker image hosted in Azure) when it was doing the analysis of a project that included Azure Bicep files. The Azure DevOps pipeline that triggered the SonarQube analysis was not failing, but within the SonarQube analysis step an error was reported in the task log INFO: Sensor IaC AzureResourceManager Sensor is restricted to changed files only INFO: 1 source file to be analyzed ##[error]ERROR: Cannot parse 'AzureServices/QueryPack.bicep:89:1' The Solution Turns out the problem was related to parsing Bicep files for App Insights Query packs. ...

The Issue This site was built using Hugo, a static site generator. I recently tried to do a long overdue update the version of Hugo from 0.108 to the current 0.122 version. I had not expected any problems, but found that the site failed to render, but with no error message all I saw was hugo.exe server --logLevel info Start building sites … hugo v0.122.0-b9a03bd59d5f71a529acb3e33f995e0ef332b3aa+extended windows/amd64 BuildDate=2024-01-26T15:54:24Z VendorInfo=gohugoio INFO copy static: syncing static files to \ INFO build: running step "process" duration "97.3263ms" INFO build: running step "assemble" duration "335.1476ms" I normally would have expected to see a INFO build: running step "render" line or an error, but got nothing, irrespective of the log level I set. ...

I was recently trying to create an Azure DevOps work item when an email is received using the Power Automate ‘Create an Azure DevOps work item when email arrives with ‘Bug’ in subject’ template. The flow created without issue, and all the drop downs were correctly populated with O365 and Azure DevOps values as expected. However, when the flow ran, on receiving an email to the correct inbox, it failed with a BadGateway error. ...

The Problem We run our SonarQube instance in a Docker container hosted in an Azure Web App Service. Today, with no notice, it failed. We did the obvious, just tried to restart it and the startup process failed. Looking at the Azure Web App’s Log Stream we could see the following error repeated on each restart attempt 2023-09-27T15:30:00.797Z INFO - Starting multi-container app.. 2023-09-27T15:30:01.024Z INFO - Pulling image: sonarqube:10.1-developer 2023-09-27T15:30:02.100Z INFO - 10.1-developer Pulling from library/sonarqube 2023-09-27T15:30:02.291Z INFO - Digest: sha256:45e7cf02e037b00028d20556a91111f8ae8ae2b2803e516cb0665dd605a6d8b2 2023-09-27T15:30:02.292Z INFO - Status: Image is up to date for sonarqube:10.1-developer 2023-09-27T15:30:02.326Z INFO - Pull Image successful, Time taken: 0 Minutes and 1 Seconds 2023-09-27T15:30:02.344Z INFO - Starting container for site 2023-09-27T15:30:02.346Z INFO - docker run -d -p 8289:9000 --name bmsonarqubeprodwebsite_sonarqube_0_ec31111b -e WEBSITES_ENABLE_APP_SERVICE_STORAGE=false -e WEBSITE_SITE_NAME=bmsonarqubeprodwebsite -e WEBSITE_AUTH_ENABLED=False -e WEBSITE_ROLE_INSTANCE_ID=0 -e WEBSITE_HOSTNAME=bmsonarqubeprodwebsite.azurewebsites.net -e WEBSITE_INSTANCE_ID=6d27706a6b4eb56feec6ef57ab9b360923c5761cabb3fb52eb6fc5f4cdfbace3 -e WEBSITE_USE_DIAGNOSTIC_SERVER=False sonarqube:10.1-developer -Dsonar.search.javaAdditionalOpts=-Dnode.store.allow_mmap=false 2023-09-27T15:30:02.347Z INFO - Logging is not enabled for this container. Please use https://aka.ms/linux-diagnostics to enable logging to see container logs here. 2023-09-27T15:31:09 No new trace in the past 1 min(s). 2023-09-27T15:32:09 No new trace in the past 2 min(s). 2023-09-27T15:33:09 No new trace in the past 3 min(s). 2023-09-27T15:33:52.465Z ERROR - multi-container unit was not started successfully The Solution Something in the back of my mind, from when we ran an on-premises SonarQube instance, made me think of a corrupt ElasticSearch index. ...

I posted a while ago about how I had automated the generation of social media posts for my static Hugo based website using Azure Logic Apps. The other place I auto-generate social media posts is from releases via my project’s Azure DevOps Pipeline builds. These use a YAML Pipeline Template that calls a Marketplace task to post to Twitter and a PowerShell task to Invoke-WebRequest to post to Mastodon. Recently the Twitter task started to fail, and given the recent changes to the Twitter API with the move to the V2 API, I decided a new solution was required. ...

Background When working with Azure DevOps, you may need to access the REST API if you wish to perform scripted tasks such as creating work items, or generating reports. Historically, you had to use a Personal Access Token (PAT) to do this. If you look in my repo of useful Azure DevOps PowerShell scripts you will find all the scripts make use of a function that creates an authenticated WebClient object using a passed in PAT token. ...

Background We use Lability to build Windows Server images for our test labs. Lability makes use of Desired State Configuration (DSC) to build the VM images. Part of this process is for Lability to download DSC modules, as ZIP files, from a NuGet feed such as PowerShell Gallery to inject into the created VM image. Historically, we have stored our own private DSC modules on an internally hosted NuGet server. However, we wanted to move these modules to a private Azure DevOps Artifacts feed. The problem was that Lability does not support downloading of DSC modules from Azure DevOps Artifact feeds, whether they are public or private, because of the way the package URLs are constructed. ...