The Issue

We have used SonarQube and the OWASP Dependency Checker Plugin for many years to perform analysis and vulnerability checking within our Azure DevOps Pipelines.

Recently, whilst picking up an old project for a new phase of development, I came across a couple of problems due to changes in both tools since the project CI/CD pipelines were last run.

  • The OWASP Dependency Checker vulnerabilities were not appearing in SonarQube as issues
  • The OWASP Dependency Checker HTML report could not (always) be loaded in SonarQube

The issues were just down to changes in both tools over time. It just goes to show that you can’t just setup a CI/CD system and expect it work forever, changes are always being introduced in cloud based tools.

The Solution

Changes in OWASP Dependency Checker supported output formats

The reason the OWASP Dependency Checker vulnerabilities were not appearing in SonarQube as Issues was because the data file format was incorrect. We were exporting XML, historically correct, when JSON is now required.

The OWASP Dependency Checker Plugin changed the way it handles importing issues in the PR Remove depreacted XML-Report Parser #755. This means the following changes are required

Viewing HTML Dependency Reports

The issue with viewing the OWASP Dependency Checker HTML report was due to size. If an HTML report that is too big has been uploaded to any branch/PR in a SonarQube project, you get Heap memory issues when you try to access the SonarQube project. This will persist until the branch/PR is removed. This will probably occur due to SonarQube’s housekeeping automation, inactive branches/PRs are removed after 30 days.

You could of course fiddle SonarQube container settings to added memory/performance, but the simple answer is to just not upload the HTML reports to SonarQube.

As long as the JSON report is uploaded all the vulnerabilities are presented within SonarQube system as issues, and as we attach the HTML report as a pipeline artifact it is still available if needed in Azure DevOps.

So the simplest answer is to not define, or comment out, the sonar.dependencyCheck.htmlReportPath SonarQube Dependency Check parameter.

The Final Pipeline

The pipeline I ended up with is as follows

 - task: SonarQubePrepare@7
    inputs:
      SonarQube: "SonarQube Service Connection"
      scannerMode: "dotnet"
      jdkversion: "JAVA_HOME_17_X64"
      projectKey: "MPK"
      projectName: "My Project Name"
      projectVersion: "$(GitVersion_Major).$(GitVersion_Minor)"
      extraProperties: |
        # Additional properties that will be passed to the scanner,
        # Put one key=value per line, example:
        sonar.dependencyCheck.jsonReportPath=$(Build.ArtifactStagingDirectory)/vulnerabilityscan/dependency-check-report.json
        # Comment out the HTML report path to avoid SonarQube heap memory issues
        sonar.dependencyCheck.htmlReportPath=$(Build.ArtifactStagingDirectory)/vulnerabilityscan/dependency-check-report.html
        sonar.cpd.exclusions=**/AssemblyInfo.cs,**/*.g.cs
        sonar.cs.vscoveragexml.reportsPaths=$(Agent.TempDirectory)/**/*.coveragexml
        sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/**/*.trx

  - task: DotNetCoreCLI@2
    displayName: ".NET Build"
    inputs:
      command: "build"
      arguments: >
        --configuration ${{ parameters.buildConfiguration }}
        --no-restore
      projects: "$(Build.SourcesDirectory)/src/MySolution.sln"

  - task: DotNetCoreCLI@2
    displayName: ".NET Test"
    inputs:
      command: "test"
      projects: "$(Build.SourcesDirectory)/src/MySolution.sln"
      arguments: >
        --configuration ${{ parameters.buildConfiguration }}
        --collect "Code coverage"
        --no-restore
        --no-build

  - task: CodeCoverage-Format-Convertor@1
    displayName: "CodeCoverage Format Converter (to allow import to SonarQube)"
    inputs:
      ProjectDirectory: "$(Agent.TempDirectory)"

  # set the version of JAVA required by the dependency-check-build-task
  - task: JavaToolInstaller@0
    inputs:
      versionSpec: '17'
      jdkArchitectureOption: 'x64'
      jdkSourceOption: 'PreInstalled'

  - task: dependency-check-build-task@6
    displayName: "Run OSWAP Vulnerability Scan"
    inputs:
      projectName: "${{ parameters.sonarQubeProjectName }}"
      scanPath: "$(Build.SourcesDirectory)/src/**"
      format: HTML, JSON
      reportsDirectory: "$(Build.ArtifactStagingDirectory)/vulnerabilityscan"
      uploadReports: false # false we publish the reports as an artifact with a name of our choice
      additionalArguments: >
        --nvdApiKey "${{ parameters.nvdApiKey }}"

  # Not needed if uploadReports: true above
  - task: PublishPipelineArtifact@1
    displayName: "Publish Vulnerability Scan Report"
    inputs:
      targetPath: "$(Build.ArtifactStagingDirectory)/vulnerabilityscan"
      publishLocation: "pipeline"
      artifactName: "${{ parameters.artifactName }}VulnerabilityScan"

  - task: SonarQubeAnalyze@7
    displayName: 'Complete the SonarQube analysis'
    inputs:
      jdkversion: "JAVA_HOME_17_X64"

  - task: SonarQubePublish@7
    displayName: 'Publish Quality Gate Result'
    inputs:
      pollingTimeoutSec: "300"

In Summary

It just shows you have to budget some time in keeping you CI/CD automation up to date on any project.