Skip to main content

References and milestones

Every result in QA-Board is compared with a reference: the web app shows each run next to the matching run of the reference, and qa gate and qa check-bit-accuracy fail on the differences. As whoever integrates a project, you choose the default reference, and the versions worth comparing with. People pick other references and save milestones in the web app.

Set the default reference​

By default, a commit is compared with the latest commit of the project's reference branch that has successful runs:

qaboard.yaml
project:
  reference_branch: main

Run your CI on that branch (see CI integration): its latest results are what every other commit is compared with. On a commit page, the reference's Change menu offers the reference branch, the milestones, and to switch the new and reference commits:

The reference commit's Change menu: a project box, Swap the bars, Also show in the top bar, Stack another reference, Remove from comparison, the reference branch main, then the milestones kd-1.3 and kd-1.2 with their batch, branch and commit.The reference commit's Change menu: a project box, Swap the bars, Also show in the top bar, Stack another reference, Remove from comparison, the reference branch main, then the milestones kd-1.3 and kd-1.2 with their batch, branch and commit.
The reference's Change menu: the reference branch, main, then the shared milestones, kd-1.3 and kd-1.2.
caution

Set reference_branch: without it, the commit page compares with the latest commit that has results on any branch, while the Change menu, qa check-bit-accuracy and qa gate use master.

List reference branches in qaboard.yaml​

Release branches and long-lived branches you often compare with can be listed under project.milestones:

qaboard.yaml
project:
  reference_branch: main
  milestones:
    - release/kd-1.3
    - quant/int8

They are listed in the Milestones menus under "From qaboard.yaml": picking one compares with the latest commit of that branch. To keep the results of one commit, save a milestone instead.

Use milestones in the CI and in scripts​

Milestones are results saved in the web app to compare with later: a commit's batch, with a name like kd-1.2, notes, and a filter. They're shared with everyone on the project, or private. See Milestones in the user guide for how people save and find them.

The sidebar's Milestones menu on the detector's commits: a search box, the favorite kd-1.3, then "My milestones" with the releases kd-1.3 and kd-1.2 and, in the group quant / calibration, entropy-try. Each shows its batch, branch, commit and date.The sidebar's Milestones menu on the detector's commits: a search box, the favorite kd-1.3, then "My milestones" with the releases kd-1.3 and kd-1.2 and, in the group quant / calibration, entropy-try. Each shows its batch, branch, commit and date.
The Milestones menu: the detector's releases, kd-1.3 and kd-1.2, shared with everyone, then a private milestone.

For integrators:

  • Gate on a release: qa gate --reference kd-1.2, or gate.reference: kd-1.2 in qaboard.yaml, compares with a shared milestone, by its name (see Quality gate). #12 names the milestone whose link is /<project>/milestone/12-..., shared or not.
  • Read them from scripts: GET /api/v1/milestones?project=<project> lists the shared milestones (and yours, with your API token): each with its label, group, notes, commit, batch and filter.
  • Link to them: each milestone has a link, /<project>/milestone/<id>-<name>, that anyone who can see the project can open. Renaming a milestone keeps its link.
  • Keep their results: the cleanup of old results keeps the batches that are milestones, and the web app won't delete all their runs.

qa check-bit-accuracy --reference takes a branch, a tag or a commit, not a milestone (see Bit accuracy).