Test pyramid

The pyramid counts test cases reported by the team’s connected GitHub repositories and saved GitLab.com project reports. Saved sprint history uses GitHub reports; GitLab contributes to Live only.

The pyramid appears on the dashboard and in the QA Metrics — Test pyramid gadget, with a Live value, a saved value for the selected closed sprint, and a trend of published sprints.

One chart combines the repositories. The contribution table and notes identify each repository, workflow, selected run, attempt, conclusion, completion time and collection time. A failed run with a valid report contributes its reported tests; the run’s conclusion does not change the JUnit outcomes.

On the dashboard

The tile shows the number of tests, how many were skipped, a small pyramid, and a line such as “240 executed · 12 Unclassified”.

The Test pyramid section shows “Tests from your pipelines, placed in layers”:

• … · shape against your targets: the rows e2e / UI, api, integration and unit, each against its target.
• Outside the pyramid, performance and synthetic and Unclassified, no layer set yet, counted beside the shape. Selecting either opens the details panel.
• Trend by sprint · tests per layer (Count) or Trend by sprint · share of each layer (Percent) for the last six sprints.
• Sources · …: a table with Repository, Where, Layers, Tests, Runs in sprint and Status.

The verdict above the shape is Top-heavy when the saved shape is inverted or the e2e / UI share is more than 2 points over its target; otherwise Off target when any layer is 5 or more points from its target; otherwise “Close to your target shape”. Without readable targets there is no verdict.

Selecting the section title opens Test pyramid details with Live, the saved sprint value, the full trend and Team goals. The sliders button opens Test pyramid settings with Team goals, Test sources and layer rules and GitLab connection.

Repository names

Newly collected reports can show the repository name observed during collection, alongside its numeric ID. This is a saved label, not a lookup of the current name or proof of its name at sprint close.

The name is visible to the same board viewers as the retained report; it does not grant GitHub access. Names do not change test totals, report selection or comparisons.

Older reports and snapshots remain unchanged and may show only Repository . Reusing an old report does not add a name retrospectively. A missing, invalid or conflicting label also keeps the numeric fallback.

When several workflows contribute, all selected reports must have a matching observed name before the repository is named.

Connect, collect and assign layers

Connect repositories in the guided setup step (see Guided setup) or in Test sources and layer rules (see GitHub repository choices). Only the default branch is read.

Collect recent runs collects completed runs created in the last 30 days, including failed and cancelled runs. Opening the page or gadget reads saved data; it does not download reports.

In Test layer rules, choose how each connected repository is classified:

• Whole repository is one layer: assign Unit, Integration, API, E2E or Outside the pyramid. This explicit choice takes precedence over individual layer rules.
• Mixed: apply layer rules: match the report’s path inside its archive, suite names, archive name, or numeric workflow ID. Removing a saved repository choice also uses Mixed.

Rules support Equals, Contains and Ends with; Directory segment is available for paths. Workflow rules use Equals with the numeric ID. Ends with can match a suite name ending in IT when the reporter emits that name.

The app does not use individual test class names or Allure labels to classify JUnit reports. For reports split by job, use their archive names; no rule assigns a job to a layer automatically.

Matching rules must agree on one layer. Conflicting matches, missing evidence that prevents a safe classification, and unmatched cases remain Unclassified. Suggested directory/suite conventions apply only when explicitly saved. The app never distributes unknown tests across layers by guess.

Outside the pyramid is for suites you intentionally keep out of the shape, such as load or synthetic checks. Their reported count and duration remain visible. Outside and Unclassified are separate rows and do not enter layer shares or goals.

GitLab reports

Select projects under GitLab projects and collect their reports explicitly. The latest saved report contributes to Live alongside GitHub reports, with its pipeline, commit and collection time.

GitLab and mixed values remain Partial because account-visible reports do not establish complete pipeline or project testing. A selected source without a usable report withholds the combined total.

Whole-project layer assignments are supported; unassigned GitLab reports stay Unclassified. Individual test names and output are not stored.

GitLab automatic collection is an optional administration setting, and GitLab reports are not saved as sprint history. See GitLab connection for access, collection and source controls.

Which GitHub reports count

For each selected run, collection reads all its archives and adds their JUnit test cases. Each reported case entry counts once. If the same test result appears in two archives, both entries count; the app does not deduplicate test names.

Parameterised cases count as the reporter lists them. Test cases omitted from the report, including filtered or commented-out tests, are unknown.

A readable archive without XML contributes no report. If no archive contains JUnit, there is no report, rather than a zero-test result. A valid JUnit report with zero cases can report zero. A missing, expired, changed or invalid archive withholds the run’s combined count.

Live selects the latest collected report separately for each workflow, then sums workflows and repositories. Latest means the maximum job completion time of the attempt, not run creation time or success status.

Copies of the same attempt collected under different rule versions count once. Reports are selected before layer classification; changing rules cannot select an older, more convenient report.

A run proven to contain no report can be skipped. An unreadable candidate that could be newer, an unknown completion time, conflicting evidence or a tied latest completion can prevent a workflow total. The app then withholds the team total and explains the missing contribution.

A complete Live value describes the collected evidence, not all GitHub history: the recent collection window can miss a run created long ago and rerun recently.

Live rules and saved sprint rules

Reports with retained suite aggregates can be reclassified without downloading their archives: Live uses the current layer rules. Older reports, or reports too large to retain suite detail, keep their collection-time summary.

If that summary’s rules do not match the requested rules, it cannot supply a newly classified total; the screen explains the limitation.

New saved sprint values select the latest report per workflow completed by the exact sprint close. A report completed before sprint start may contribute, with its actual run and date shown. Runs created before sprint start are included in preparation.

The layer rules effective at the close apply, even if the reports are collected after a later rule change.

When a sprint closes, the app attempts to publish its value. If collection is incomplete, the sprint remains unpublished. In Test pyramid details, Prepare this sprint requests background collection of available history and preparation; use Reload to check progress. Anyone who can see the team’s board can request preparation.

Preparation checks the available runs, including queued/running attempts and archived earlier attempts. It cannot recover deleted provider history. Missing earlier-attempt evidence gives Partial without a team sum; repairable missing current evidence keeps preparation unpublished. The caption describes the available history observed during preparation.

Published values are immutable. Changing rules, collecting again or changing reference goals does not rewrite them. Older per-repository snapshots keep their original selection; they are not split into invented workflow contributions.

If Jira’s sprint dates change after publication, the block reports the changed window instead of presenting the saved value as a new close.

A report is permanently unavailable

For a newly prepared sprint, a verified invalid JUnit report or an explicitly expired artifact can publish a Partial value. The app counts the last confirmed archived report of the same repository, default branch and workflow completed before the unavailable run and sprint close.

It names both the older counted run and the newer unavailable run. An archived report can still be used after its run disappears from GitHub; this does not reconstruct deleted GitHub history.

Fix the JUnit output for future runs, or increase artifact retention for future reports. If no compatible archive exists, the layers and team total remain unavailable, not zero.

Conflicting identities, uncertain completion, tied latest reports, incomplete attempts or changed sources never permit this fallback. An empty artifact list alone does not prove that retention expired.

This saved Partial value is final: Prepare is no longer offered and later collection does not rewrite the sprint. Live continues to use its normal latest-report rules. A Partial sprint is not used for a historical delta or a complete trend point.

Counts, percentages and warnings

Every layer keeps its count and reported duration, including empty target layers.

Display	Calculation
Reported count	Passed + failed + errors + skipped
Executed count	Passed + failed + errors
Layer percentage	Executed cases in that layer / executed cases across Unit + Integration + API + E2E
Skipped share	Skipped cases / all reported cases, including Outside and Unclassified
Goal difference	Actual executed share minus the target, in percentage points

In Test pyramid details, Count / Percent changes the saved value, Live and trend together. Count bars use a solid executed segment and a hatched skipped segment; percent bars show executed layer shares. Outside and Unclassified remain counts.

With no classified executed cases, layer percentages, shape and goal comparisons are unavailable. With no reported cases, skipped share is unavailable.

For example, a report with 10 passed, 1 failed and 1 skipped case has 12 reported cases and 11 executed cases. Its skipped share is 1 / 12 = 8.3%. If all cases are Unclassified, the app still shows 12; it does not invent a layer percentage.

Durations remain the times reported for all cases, with missing durations identified; no executed-only duration is inferred.

The headline in Test pyramid details is the reported classified count, including classified skipped cases. Its delta compares the previous published sprint under the same layer rules.

No comparison is shown before the first published sprint, across changed rules, for an unavailable total, or beyond the twelve-sprint trend boundary. These cases carry explanations.

The trend uses executed layer counts or shares, with a separate Skipped (all cases) column. Outside and Unclassified remain reported counts. Each saved point keeps its rules version; Partial points with no team total keep their place and explanation.

Team goals

Anyone who can see the team’s board can open Team goals in Test pyramid settings or in Test pyramid details. Goal preset offers Pyramid (50 / 25 / 15 / 10), Trophy (20 / 45 / 25 / 10) or Custom.

The four targets are Unit target (%), Integration target (%), API target (%) and End-to-end target (%). “The four targets must total 100%.” Editing a target switches the preset to Custom; the presets are editable starting points.

Targets use classified executed tests (passed, failed and errors), excluding skipped, Outside and Unclassified. The dashboard section compares all four layers. Test pyramid details and the gadget compare three groups: Unit, Integration + API (the sum of the two targets) and End to end.

The default goal is the Pyramid preset. Goals saved before the four targets existed had one middle target; they open as Custom, with the middle target split equally between Integration and API.

An inverted-shape warning appears when executed E2E cases exceed End-to-end warning threshold (%) (20% by default) or outnumber executed Unit cases. A skipped warning appears when the skipped share exceeds Skipped warning threshold (%) (10% by default).

Equality with a threshold does not trigger a warning. Targets accept steps of 0.005%, thresholds steps of 0.01%.

Save team goals applies only a confirmed save. The gadget reads the same saved preferences. If a save fails or another editor changes them, use Reload team goals before editing again.

These are current reference goals, also when viewing an older sprint. Changing them does not change measured counts, durations, classification or historical deltas. If goals cannot be read, measured data remain visible while goal comparisons and warnings are withheld.

When there is no team total

A missing value is not zero. Read the workflow and repository notes to identify the reason.

State or reason	Meaning
No repository connected	Connect a source before collecting.
Being prepared / no saved pyramid	There is no published value yet. Use Prepare this sprint in Test pyramid details when offered, then Reload.
Available history could not be confirmed	Preparation could not establish enough current evidence; an unpublished sprint can be prepared again.
Earlier attempts are not archived	Available evidence cannot establish the historical winner.
No test report / report unavailable	A report is absent, or a potentially newer report could not be read.
Completion uncertain / latest report cannot be determined	Timing or conflicting evidence cannot establish one latest report.
Branch unknown / source changed	Provenance is missing or differs from the repository’s current default branch.
Suite detail unavailable under these rules	A fixed older summary cannot be reclassified under the requested rules.
Collection or size limit	Available history or totals exceed supported bounds; no truncated total is shown.
Sprint dates changed	The saved value belongs to the original close and is not recalculated.

A published Partial value is also immutable; collecting again does not replace it. Older snapshots may show a legacy reason such as “no run on the default branch in this sprint, carried forward from sprint N.” in details; the dashboard section shows “Counted from sprint N”. That legacy carry-forward does not describe the new per-workflow selection.

Dashboard gadget and limits

Add QA Metrics — Test pyramid from Jira’s gadget catalogue. In Edit, choose a team, a closed sprint and a title. The gadget displays saved values and current goals; it cannot collect or prepare a sprint. Use Test pyramid details on the QA Metrics page for those actions.

The compact gadget shows one observation at a time: Original result, an available Recollected result, Live, or Published sprint trend. Its primary number is total reported cases, with executed and skipped totals beside it.

The shape always uses classified executed shares; Count changes layer labels to reported counts, while Percent labels show executed shares. Outside, Unclassified and skipped remain separate from the shape denominator.

Expand Sources, report evidence and current goals for all six layer counts, durations, retained workflow/run evidence and team goals. Warnings and unavailable states remain visible without opening it.

The compact trend shows original published values only and keeps gaps and changed-rule boundaries. Select a period to inspect its retained layer values.

Open pyramid details opens the QA Metrics page with Test pyramid details at the same sprint, result and units. A recollected link identifies the exact attempt; if that attempt is no longer available, the page explains this instead of substituting another.

Manual selection does not overwrite the gadget defaults. The compact gadget explains why a delta is unavailable: its saved trend does not establish comparable source, workflow and sprint-window provenance. Live and recollected values are not substituted into the original trend.

• GitHub collection runs on the configured schedule or can be requested explicitly. GitLab supports explicit or scheduled collection into Live only. See Automatic collection below.
• A team supports up to 40 connected repositories. Live reads up to 1,000 archived outcomes per repository across rule versions. Sprint inventory is bounded to 999 runs per repository and 40 pages across its two passes, with additional time, request and snapshot-size limits.
• A run supports up to 16 archives, 2 MiB combined compressed data, 64 XML files and 100,000 cases, with additional ZIP safety limits. Exceeding a limit withholds the combined result.
• Suite aggregates are kept for up to 2,048 distinct artifact/path/suite-chain combinations and 48 KB per run. Larger reports keep a fixed summary. Test case names, individual class names and failure output are not stored.
• Recollection after a default-branch change creates a separate result; see Recollect after the default branch changes. An empty artifact list alone cannot establish expiration; legacy failures without fresh verification keep their original meaning.

Automatic collection

Connected GitHub repositories are collected automatically. In Administration (shown to users with the Administer Jira global permission), under Automatic report collection, GitHub collection interval offers Every 6 hours (default), Every 12 hours, Every 24 hours or Paused.

This installation setting applies to all teams; board access still controls team sources and layer rules. Pausing stops new automatic requests; already saved requests finish through background recovery. Explicit collection remains available.

GitHub visits up to five repositories per hour and remembers its position. Large installations may collect later than the selected minimum interval. A running collection is not replaced.

The supported window is runs created on the default branch during the last 30 days. Repeated scans reuse compatible archived reports; they still verify run identity and completion. Older-created reruns are outside automatic discovery. More than 999 matching runs remains an explicit limit, not a complete or zero result.

Opening a page or gadget only reads saved results. Collect GitHub reports shows the creation window and saved reports. An unchanged archived report keeps its original collection timestamp; a new schedule visit does not manufacture a newer report observation. Published sprint snapshots remain unchanged.

GitLab has its own GitLab collection interval in the same place, initially Paused. It visits up to two selected GitLab projects per hour and keeps its cursor; the interval is a minimum between attempts, including failures. It may renew the saved connection when access expires.

Pause stops new GitLab attempts; an already claimed report may finish while its source and connection remain valid. GitLab and mixed Live remain Partial. No GitLab sprint history is created. Selecting a project alone does not enable this schedule.

Set up report groups

In Test sources and layer rules, expand Test layer rules. Set up GitHub test reports walks through three steps: publish JUnit, collect reports from a saved repository, then choose an observed group and a layer.

Choose a saved repository and Load observed report groups. This reads artifact names and workflow IDs from saved reports. It does not scan GitHub, infer job names, or prove that an artifact is still available.

Uncollected, unavailable and older reports without group detail have separate explanations. A valid empty report remains a report with zero cases.

Choose Load layer rules, an observed group and a layer, then Add observed group to draft. Review the exact-match rule and Save layer rules. An existing exact rule is not replaced.

Artifact-name rules apply to all mixed repositories in the team; a workflow ID matches that workflow. Whole-repository assignments take precedence. Conflicting layer matches remain Unclassified.

Saved reports with suite detail follow the new rule on the next Live read without collection; published sprint snapshots keep their original rules.

Publish JUnit reports includes GitHub Actions and GitLab CI snippets. Configure your test runner to generate JUnit XML first, adapt the file glob, and add the upload step or artifacts block to the existing test job. Uploading does not generate XML or connect a source.

Keep reports until collection finishes. A failed job can still provide a valid test report. Publication syntax checked on 2026-09-20 against upload-artifact, GitHub always() and GitLab unit test reports.

Recollect after the default branch changes

A saved sprint keeps the source that was used at publication. After a later collection confirms a different default branch, the page and gadget label that result as Original. They do not contact GitHub when rendering this notice.

A user with the Administer Jira global permission and access to the team’s board can choose Recollect using the new source in Test pyramid details. Other users see “A site administrator can recollect this sprint from the current default branch.” Recollection creates a separate request and result; the original snapshot and sprint trend remain unchanged.

The request pins the current branch and collects available runs again. Only attempts whose last job finished between the original sprint start and close can count, including reruns created earlier. Earlier reports are not carried into this new result.

Missing completion, incomplete attempt history, a changed source or provider limits remain explained gaps. If no available attempt can be confirmed inside the window, the source-change explanation remains instead of showing a zero. Deleted provider history cannot be reconstructed.

Reload to check progress, then choose Original result or Recollected result. The latter shows its own publication time and provenance. A failed request does not alter any saved number; another manual request may retry it.

At most 20 manual recollections are kept per sprint; duplicate clicks while one request is pending reuse that request. No branch change starts recollection automatically. Background recovery only resumes an already requested job.

Missing artifacts with an unknown cause

An empty artifact listing does not tell whether a run never uploaded reports or whether someone removed them later. It does not prove expiry.

If that run could replace the latest counted report, the pyramid shows Partial and withholds the total instead of silently showing an older number as Complete. Successfully read archives with no JUnit remain different from unavailable artifacts. A valid JUnit report containing zero tests is still a real zero.

An older unavailable run stops blocking a newer report only when the saved evidence proves that it completed earlier. Saved sprint results are not rewritten by later observations.