Problem

I recently tripped myself up whist adding SonarQube analysis to a rather complex Azure DevOps build.

The build has two VsTest steps, both were using the same folder for their test result files. When the first VsTest task ran it created the expected .TRX and .COVERAGE files and then published its results to Azure DevOps, but when the second VsTest task ran it over wrote this folder, deleting the files already present, before it generated and published it results.

This meant that the build itself had all the test results published, but when SonarQube looked for the files for analysis only the second set of test were present, so its analysis was incorrect.

Solution

The solution was easy, use different folders for each set of test results.

This gave me a build, the key items are shown below, where one VsTest step does not overwrite the previous results before they can be processed by any 3rd party tasks such as SonarQube.

steps:  
- task: SonarSource.sonarqube.15B84CA1-B62F-4A2A-A403-89B77A063157.SonarQubePrepare@4  
   displayName: 'Prepare analysis on SonarQube'  
   inputs:  
     SonarQube: Sonarqube  
     projectKey: 'Services'  
     projectName: 'Services'  
     projectVersion: '$(major).$(minor)'  
     extraProperties: |  
      # Additional properties that will be passed to the scanner,   
      sonar.cs.vscoveragexml.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.coveragexml  
      sonar.cs.vstest.reportsPaths=$(System.DefaultWorkingDirectory)/\*\*/\*.trx

  

… other build steps

  

\- task: VSTest@2  
   displayName: 'VsTest – Internal Services'  
   inputs:  
     testAssemblyVer2: |  
      \*\*\*.unittests.dll  
      !\*\*obj\*\*  
     searchFolder: '$(System.DefaultWorkingDirectory)/src/Services'  
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsServices'  
     overrideTestrunParameters: '-DeploymentEnabled false'  
     codeCoverageEnabled: true  
     testRunTitle: 'Services Unit Tests'  
     diagnosticsEnabled: True  
   continueOnError: true

\- task: VSTest@2  
   displayName: 'VsTest - External'  
   inputs:  
     testAssemblyVer2: |  
      \*\*\*.unittests.dll  
      !\*\*obj\*\*  
     searchFolder: '$(System.DefaultWorkingDirectory)/src/ExternalServices'  
     resultsFolder: '$(System.DefaultWorkingDirectory)TestResultsExternalServices'  
     vsTestVersion: 15.0  
     codeCoverageEnabled: true  
     testRunTitle: 'External Services Unit Tests'  
     diagnosticsEnabled: True  
   continueOnError: true  

\- task: BlackMarble.CodeCoverage-Format-Convertor-Private.CodeCoverageFormatConvertor.CodeCoverage-Format-Convertor@1  
   displayName: 'CodeCoverage Format Convertor'  
   inputs:  
     ProjectDirectory: '$(System.DefaultWorkingDirectory)'  

\- task: SonarSource.sonarqube.6D01813A-9589-4B15-8491-8164AEB38055.SonarQubeAnalyze@4  
   displayName: 'Run Code Analysis'

\- task: SonarSource.sonarqube.291ed61f-1ee4-45d3-b1b0-bf822d9095ef.SonarQubePublish@4  
   displayName: 'Publish Quality Gate Result'