Skip to main content

Finding results

Projects list​

The home page lists all projects that have results.

  • Search with the filter projects... box. It looks at all projects, even those not shown yet.
  • Pin a project at the top of the list with its star. Favorites are remembered by your browser only.
  • Projects are sorted by favorites first, then by their latest run. Each card shows when the last run (or commit) happened and how many commits have results.
  • The first 50 projects show up first, more come as you scroll down (or click Show more).
  • Source opens the project's git repository.

Click a project to open its commit list.

The QA-Board home page: a filter box, then a card per Kitefin project with its avatar, name, description, a star to pin it, a link to its source, when it last ran and how many commits have results.
Each card says when the project last ran and how many commits have results. Starred projects stay on top.
tip

Project names are git paths, like group/repo. In monorepos, each sub-project has its own entry, like group/repo/sub-project.

Commit list​

The project's home page lists the latest commits with results, grouped by day ("Today — 3 commits"). It refreshes every minute. The 50 latest commits show up first, more come as you scroll down (or click More commits). On a branch page, all the branch's commits in the date range come at once (up to 1000), for the metrics chart. At the end of the date range, Older commits looks 7 more days back.

The inference gateway's commits from all branches, grouped by day: each with its message, author, branch, the median and p99 of its CI batch, and "1 crashed" in red on the batcher change that panics under bursts.
Each row says how its CI batch went: the main metric's median when it finished, a crashed batch in red, how many other batches it ran.

Each commit row shows:

WhereWhatClick it to...
AvatarThe committersee all their commits
Message, short hashThe commitopen the commit diff on your git server (hash), or copy the full hash (copy icon)
Star iconThe commit is a milestone
Menu icon (☰)Copy the commit's directory (Windows or Linux path), view its files in the browser, delete its runs
Branch nameThe commit's branchsee the branch's commits
"2 hours ago by ..."Commit date and committersee the committer's commits
Right sideThe CI batch status: pending, running, crashed runs, the main metric (median and average), ... for all aggregated metrics
N other batchesHover to list the commit's other batches (tuning, local runs...) and jump to one
Green N CI results buttonopen the commit page
note

No results or Check the pipeline status... means the commit has no finished runs yet in its CI batch: check your CI pipeline.

Branches and committers​

  • The sidebar shows the project's reference branch: click it to see only its commits.
  • Click any branch name in a commit row to see that branch. The URL is /<project>/commits/<branch>.
  • On a branch page, a chart of a main metric over time comes first: for each commit, the median (or average) over its runs, one line per batch (CI, nightly...). Pick the metric and the aggregation above the chart. Hover a point to see the commit and its reference, click to freeze the reference. For more, use History.
  • Click an avatar or a committer name to see their commits, on all branches: /<project>/committer/<name>.
Above main's commits, a chart of the median p99 of each commit's CI batch over six weeks, falling from over 10 ms to a few. The point of the int8 commit is hovered: below the chart, it shows as New, the commit before it as Reference.
Above main's commits, each point is a commit's median latency over its runs. Hover a point to see its commit and the reference it's compared with.

Searching commits​

The Filter... box at the top right searches all the project's commits, not only those in the date range (on a branch page, all the branch's commits). Results come as you pause typing, newest first, with a "Searching all commits" banner that counts the matches. When your search could be a branch name (a single word, or a branch: term), it also suggests the most recently active branches whose name contains it: pick one to open it.

You typeYou find commits...
denoisewhose message, committer, branch or batch labels contain "denoise"
3fa2c1the same, or whose hash starts with it, for words of 4 or more hexadecimal characters
denoise crashmatching both words
-wipnot matching "wip"
"fix the crash"containing these words together
branch:featurewhose branch contains "feature"
committer:alice or author:aliceby a committer whose name contains "alice"
batch:tuning or label:tuningwith a batch whose label contains "tuning"
message:speed or msg:speedwhose message contains "speed"
id:3fa2, commit:3fa2 or sha:3fa2whose hash starts with "3fa2"

Searches are case-insensitive, and qualifiers can be excluded too (-branch:release). Unlike the filters of the commit page, they don't use regular expressions. The search is saved in the URL (search=), so a shared link shows the same results.

The filter box of the commits page with "cache" typed in: it suggests the agent/result-cache and perf/sharded-cache branches, and "Go to commit: cache"; below, only the commits that mention the cache.
Typing suggests the branches that match, and offers to go straight to the commit with that hash.

Date range​

By default, the list shows the last 3 days. Change the range with the date picker at the top left (it has shortcuts like "past week"), and use the refresh button to reload.

tip

"Could not find a commit with results" usually means you should widen the date range.

Going to a commit by its hash​

  • Type or paste a commit hash in the Filter... box, then choose Go to commit: ....
  • Or edit the URL: /<project>/commit/<hash>. Short hashes work.
  • On a commit page, you can also type a hash in the commit field of the new or reference bar (see Comparing results).

Milestones​

The sidebar's Milestones menu lists the project's milestones: those from qaboard.yaml, the shared ones, and your private ones. Pick one to open it. See Milestones.

The sidebar's Milestones menu on the detector's commits: the shared milestones kd-1.3 and kd-1.2, each with its branch, commit and author, then a private milestone, entropy-try, on the quant/entropy branch.
The releases are shared milestones, for everyone; the private one is only in your browser.

Batches​

A commit can have many batches. On the commit page, the batch selector next to the filter box shows the selected batch and lets you switch:

The new commit's batch selector open: the CI batch, run by ci, 18 of 18 runs successful; ines 🏠 ines-halo, Ines's local runs, 5 of 5; and the tune-denoise tuning experiment, 60 of 60.
The CI batch, someone's local runs (🏠) and a tuning experiment, each with how many of its runs succeeded.
  • CI is the batch from your continuous integration (label default).
  • user 🏠 label are local runs started by user from their workspace. A warning reminds you they may not use the code under version control.
  • Other labels are tuning experiments or runs started with qa --label.
  • Each entry shows successful/total ✅, crashed ❌, running 🏃, optimization iterations 🔁, a ⭐ for milestones, and who started it.
  • The list is searchable: type a label, a user name, or words like failed, running, local, ci, tuning.
  • The list opens on the selected batch, which has a check mark. Use ↑ ↓ and Enter to pick another one.

The new and reference bars each have their own batch selector, so you can compare a tuning batch with the CI batch of another commit.

Filter syntax​

The filter new outputs and filter ref outputs boxes on the commit page keep only matching runs. The counters below them (8 outputs, 4 crashed...) count what's left.

You typeYou keep runs...
nightwhose input path, configuration, platform, metadata, tuning parameters or batch contain "night"
night indoormatching both words
-nightnot matching "night"
2X5|GW1matching a regular expression
threshold:0.5with tuning parameter threshold equal to 0.5 (also threshold=0.5)
fail or crashthat crashed
pending or runningthat are not finished
id:1234with this run id
The new commit's filter box with "low-light -gradient", 8 outputs under it, and the Metrics Table below listing only the low-light runs, the sky gradient left out.
The filter “low-light -gradient” keeps the low-light runs except the sky gradient's: the counter under the box and the Metrics Table only count what's left.

Filters are case-insensitive and are saved in the URL (filter= and filter_ref=), so a shared link shows the same runs.