Running Microsoft Test Manager Test Suites as part of a vNext Release pipeline - Part 2

In my last post I discussed how you could wire TCM tests into a Release Management vNext pipeline. The problem with the script I provided, as I noted, was that the deployment was triggered synchronously by the build i.e. the build/release process was:

TFS Build
1. Gets the source
2. Compiled the code
3. Run the unit tests
4. Trigger the RM pipeline
5. Wait while the RM pipeline completed
RM then
1. Deploys the code
2. Runs the integration tests
When RM completed the TFS build completes

This process raised a couple of problems

You cannot associate the integration tests with the build as TCM only allow association with completed successful builds. When TCM finishes in this model the build is still in progress.
You have to target only the first automated stage of the pipeline, else the build will be held as ‘in progress’ until all the release stages have complete, which may be days if there are manual approvals involved

The script InitiateReleaseFromBuild

These problems can all be fixed by altering the PowerShell that triggers the RM pipeline so that it does not wait for the deployment to complete, so the TFS build completes as soon as possible.

This is done by passing in an extra parameter which is set in TFS build

param(  
    \[string\]$rmserver = $Args\[0\],  
    \[string\]$port = $Args\[1\],    
    \[string\]$teamProject = $Args\[2\],     
    \[string\]$targetStageName = $Args\[3\],  
    \[string\]$waitForCompletion = $Args\[4\]  
) 

cls  
$teamFoundationServerUrl = $env:TF\_BUILD\_COLLECTIONURI  
$buildDefinition = $env:TF\_BUILD\_BUILDDEFINITIONNAME  
$buildNumber = $env:TF\_BUILD\_BUILDNUMBER

 

  
"Executing with the following parameters:\`n"  
"  RMserver Name: $rmserver"  
"  Port number: $port"  
"  Team Foundation Server URL: $teamFoundationServerUrl"  
"  Team Project: $teamProject"  
"  Build Definition: $buildDefinition"  
"  Build Number: $buildNumber"  
"  Target Stage Name: $targetStageName\`n"  
"  Wait for RM completion: $waitForCompletion\`n"

 

$wait = \[System.Convert\]::ToBoolean($waitForCompletion)  
$exitCode = 0

 

trap  
{  
  $e = $error\[0\].Exception  
  $e.Message  
  $e.StackTrace  
  if ($exitCode -eq 0) { $exitCode = 1 }  
}

 

$scriptName = $MyInvocation.MyCommand.Name  
$scriptPath = Split-Path -Parent (Get-Variable MyInvocation -Scope Script).Value.MyCommand.Path

 

Push-Location $scriptPath    

 

$server = \[System.Uri\]::EscapeDataString($teamFoundationServerUrl)  
$project = \[System.Uri\]::EscapeDataString($teamProject)  
$definition = \[System.Uri\]::EscapeDataString($buildDefinition)  
$build = \[System.Uri\]::EscapeDataString($buildNumber)  
$targetStage = \[System.Uri\]::EscapeDataString($targetStageName)

 

$serverName = $rmserver + ":" + $port  
$orchestratorService = "[http://$serverName/account/releaseManagementService/\_apis/releaseManagement/OrchestratorService"](http://$serverName/account/releaseManagementService/_apis/releaseManagement/OrchestratorService")

 

$status = @{  
    "2" = "InProgress";  
    "3" = "Released";  
    "4" = "Stopped";  
    "5" = "Rejected";  
    "6" = "Abandoned";  
}

 

$uri = "$orchestratorService/InitiateReleaseFromBuild?teamFoundationServerUrl=$server&teamProject=$project&buildDefinition=$definition&buildNumber=$build&targetStageName=$targetStage"  
"Executing the following API call:\`n\`n$uri"

 

$wc = New-Object System.Net.WebClient  
$wc.UseDefaultCredentials = $true  
\# rmuser should be part rm users list and he should have permission to trigger the release.

 

#$wc.Credentials = new-object System.Net.NetworkCredential("rmuser", "rmuserpassword", "rmuserdomain")

 

try  
{  
    $releaseId = $wc.DownloadString($uri)

 

    $url = "$orchestratorService/ReleaseStatus?releaseId=$releaseId"

 

    $releaseStatus = $wc.DownloadString($url)

 

  
    if ($wait -eq $true)  
    {  
        Write-Host -NoNewline "\`nReleasing ..."

 

        while($status\[$releaseStatus\] -eq "InProgress")  
        {  
            Start-Sleep -s 5  
            $releaseStatus = $wc.DownloadString($url)  
            Write-Host -NoNewline "."  
        }

 

        " done.\`n\`nRelease completed with {0} status." -f $status\[$releaseStatus\]  
    } else {

 

        Write-Host -NoNewline "\`nTriggering Release and exiting"  
    }

 

}  
catch \[System.Exception\]  
{  
    if ($exitCode -eq 0) { $exitCode = 1 }  
    Write-Host "\`n$\_\`n" -ForegroundColor Red  
}

 

if ($exitCode -eq 0)  
{  
    if ($wait -eq $true)  
    {  
        if ($releaseStatus -eq 3)  
        {  
          "\`nThe script completed successfully. Product deployed without error\`n"  
        } else {  
            Write-Host "\`nThe script completed successfully. Product failed to deploy\`n" -ForegroundColor Red  
            $exitCode = -1 # reset the code to show the error  
        }  
    } else {  
        "\`nThe script completed successfully. Product deploying\`n"  
    }  
}  
else  
{  
  $err = "Exiting with error: " + $exitCode + "\`n"  
  Write-Host $err -ForegroundColor Red  
}

 

Pop-Location

 

exit $exitCode

The Script TcmExecWrapper

A change is also required in the wrapper script I use to trigger the TCM test run. We need to check the exit code from the inner TCM PowerShell script and update the TFS build quality appropriately.

To this I use the new REST API in TFS 2015 as this is far easier than using the older .NET client API. No DLLs to distribute.

It is worth noticing that

I pass the credentials into the script from RM that are used to talk to the TFS server. This is because I am running my tests in a network isolated TFS Lab Environment, this means I am in the wrong domain to see the TFS server without providing login details. If you are not working cross domain you could just use Default Credentials.
RM only passes the BuildNumber into the script e.g. MyBuild_1.2.3.4, but the REST API need the build id to set the quality. Hence the need for function Get-BuildDetailsByNumber to get the id from the name

\# Output execution parameters.  
$VerbosePreference ='Continue' # equiv to -verbose  
function Get-BuildDetailsByNumber  
{  
    param  
    (  
        $tfsUri ,  
        $buildNumber,  
        $username,   
        $password  
    )  
    $uri = "$($tfsUri)/\_apis/build/builds?api-version=2.0&buildnumber=$buildNumber"  
    $wc = New-Object System.Net.WebClient  
    #$wc.UseDefaultCredentials = $true  
    $wc.Credentials = new-object System.Net.NetworkCredential($username, $password)  
      
    write-verbose "Getting ID of $buildNumber from $tfsUri "  
    $jsondata = $wc.DownloadString($uri) | ConvertFrom-Json   
    $jsondata.value\[0\]  
    
}  
function Set-BuildQuality  
{  
    param  
    (  
        $tfsUri ,  
        $buildID,  
        $quality,  
        $username,   
        $password  
    )  
    $uri = "$($tfsUri)/\_apis/build/builds/$($buildID)?api-version=1.0"  
    $data = @{quality = $quality} | ConvertTo-Json  
    $wc = New-Object System.Net.WebClient  
    $wc.Headers\["Content-Type"\] = "application/json"  
    #$wc.UseDefaultCredentials = $true  
    $wc.Credentials = new-object System.Net.NetworkCredential($username, $password)  
      
    write-verbose "Setting BuildID $buildID to quality $quality via $tfsUri "  
    $wc.UploadString($uri,"PATCH", $data)   
      
}  
$folder = Split-Path -Parent $MyInvocation.MyCommand.Definition  
write-verbose "Running $folderTcmExecWithLogin.ps1"   
& "$folderTcmExecWithLogin.ps1" -Collection $Collection -Teamproject $Teamproject -PlanId $PlanId  -SuiteId $SuiteId -ConfigId $ConfigId -BuildDirectory $PackageLocation -TestEnvironment $TestEnvironment -LoginCreds "$TestUserUid,$TestUserPwd" -SettingsName $SettingsName -BuildNumber $BuildNumber -BuildDefinition $BuildDefinition  
write-verbose "Got the exit code from the TCM run of $LASTEXITCODE"  
$url = "$Collection/$Teamproject"  
$jsondata = Get-BuildDetailsByNumber -tfsUri $url -buildNumber $BuildNumber -username $TestUserUid -password $TestUserPwd  
$buildId = $jsondata.id  
write-verbose "The build ID is $buildId"  
$newquality = "Test Passed"  
if ($LASTEXITCODE -gt 0 )  
{  
    $newquality = "Test Failed"  
}  
   
write-verbose "The build quality is $newquality"  
Set-BuildQuality -tfsUri $url  -buildID $buildId -quality $newquality -username $TestUserUid -password $TestUserPwd

Note: TcmExecWithLogin.ps1 is the same as in the In my last post

Summary

So with these changes the process is now

TFS Build
1. Gets the source
2. Compiled the code
3. Run the unit tests
4. Trigger the RM pipeline
5. Build ends
RM then
1. Deploys the code
2. Runs the integration tests
3. When the test complete we set the TFS build quality

This means we can associate both unit and integration tests with a build and target our release at any stage in the pipeline, it pausing at the points manual approval is required without blocking the initiating build.

The script InitiateReleaseFromBuild#

The Script TcmExecWrapper#

Summary#

The script InitiateReleaseFromBuild

The Script TcmExecWrapper

Summary