Skip to content

Union code coverage across all six pipelines#3716

Open
aaronburtle wants to merge 13 commits into
mainfrom
dev/aaronburtle/combined-coverage-union
Open

Union code coverage across all six pipelines#3716
aaronburtle wants to merge 13 commits into
mainfrom
dev/aaronburtle/combined-coverage-union

Conversation

@aaronburtle

Copy link
Copy Markdown
Contributor

Why make this change?

Closes #3715

What is this change?

  • Adds a best-effort Total Code Coverage job to the existing unit-test
    pipeline (unittest-pipelines.yml) that surfaces the true combined coverage
    across all six test pipelines, instead of the misleadingly low per-pipeline
    numbers.
    • DAB's tests are split across six independent ADO pipelines (unit + MsSql,
      PostgreSql, MySql, DwSql, CosmosDb). Each publishes its own coverage tab and
      nothing merges them, so every pipeline in isolation reads much lower than the union.
  • How it works:
    • The build job publishes this run's unit cobertura as a coverage-unit
      pipeline artifact.
    • The new total_code_coverage job (dependsOn: build) downloads that plus the
      latest available cobertura from the five DB pipelines
      (DownloadPipelineArtifact@2, definition IDs DwSql=1, MsSql=2, MySql=3,
      PgSql=4, Cosmos=6).
    • scripts/merge-coverage.ps1 unions all reports and emits one combined
      Cobertura.xml, which PublishCodeCoverageResults@2 publishes.
  • Design notes:
    • Rolling dashboard, not a synchronized per-PR gate. DB coverage is pulled
      with runVersion: latest, so it may reflect an older commit ADO offers no
      primitive to block on all six sibling runs from inside one of them. A true
      per-PR guarantee would require consolidating all suites into one pipeline with
      a dependsOn-all aggregation job. This is not necessary for tracking code coverage.
    • Best-effort: continueOnError on the job and every step, so it can never
      fail or block unit CI. If aggregation fails, the build job's own unit-only
      coverage remains as the tab fallback.
    • The merge keys on fully-qualified class name + line (agent-path independent)
      so points union correctly across Windows/Linux agents, and excludes *.Tests
      assemblies.

How was this tested?

  • Integration Tests
  • Unit Tests

Validated scripts/merge-coverage.ps1 locally against real cobertura from all six
pipelines (combined = 76.77% branch / 83.33% line) and against synthetic reports
covering cross-agent path union, branch-condition max-merge, test-assembly
exclusion, and well-formed Cobertura output. Pipeline changes are CI-only.

Sample Request(s)

  • N/A — CI/pipeline change only; no REST, GraphQL, or CLI surface is affected.
  • Local usage of the merge script:
    ./scripts/merge-coverage.ps1 -Path <folder-of-cobertura> -OutFile combined/Cobertura.xml
    

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a “Total Code Coverage” aggregation mechanism to provide a more accurate combined (union) coverage number across DAB’s six independent Azure DevOps pipelines by downloading each pipeline’s Cobertura output and merging them into a single report.

Changes:

  • Introduces scripts/merge-coverage.ps1, a dependency-free Cobertura union merger that can also emit a merged Cobertura XML for publishing.
  • Updates the unit-test pipeline to publish its coverage as a pipeline artifact and adds a new best-effort job that downloads latest DB-pipeline coverage and publishes a combined coverage report.
  • Extends the shared build/unit-test template to stage and publish the unit coverage artifact used by the aggregation job.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File Description
scripts/merge-coverage.ps1 Adds a PowerShell Cobertura union-merger (line + branch) and optional merged XML emitter.
.pipelines/unittest-pipelines.yml Converts the unit pipeline to multi-job and adds a best-effort “Total Code Coverage” aggregation job.
.pipelines/templates/build-pipelines.yml Publishes unit Cobertura files as a pipeline artifact (coverage-unit) for later aggregation.

Comment thread .pipelines/unittest-pipelines.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Test related issue.

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

Aggregate code coverage reports to identity total code coverage

5 participants