The commit page

A commit page shows the results of one commit, the new one, next to a reference: by default, the latest results of your project's reference branch. It's where you answer "what did this commit change?". This page shows you around.

1

2

3

4

The two bars say what you compare: here, a merge into main against the release before it. The sidebar picks the view; the View options, on the right, what every run's card shows.

1. 1The new commit
2. 2The reference
3. 3The views
4. 4View options

To review a commit, go from the big picture down to the pixels:

1. Summary: did the main metrics get better or worse? See Reading metrics.
2. Metrics Diff: which runs changed, and by how much?
3. Visualizations: look at those runs' outputs next to the reference's.

For a guided tour on the demo data, follow Your first review.

The two bars

The new bar, on top, holds the commit you look at; the ref bar, below, what you compare it with. They work the same way. Each shows:

• its name, new or ref: click it to rename what you compare, e.g. ISP v2 and ISP v1. See Name what you compare;
• the commit message, author and date;
• the commit hash: type another one to switch commits, or click Change to pick a branch, a milestone, or to swap the bars. See choose another reference;
• the branch: click it to select its latest commit;
• a star, to save the results you see (this batch, with its filter) as a milestone;
• a filter box, to keep only some runs, and the counters of runs, running, pending and failed below it. See the filter syntax;
• the batch selector: the CI batch, your local runs, tuning experiments... See Batches;
• a link button on the new bar, to copy a link to the page that pastes as its name in chats, see Sharing; a refresh button, and a menu (☰) with the commit's and the batch's folders and actions.

To compare with several references, like a release and the latest main, hover the ref bar and click Stack a reference: tabs then show below the bars, one per reference, and r switches between them. See Compare with several references.

The Logs, Available Tests and Run Tests / Tuning views only concern the new commit: they hide the reference bar. While runs are pending, the page refreshes by itself every 15 seconds.

Every item of the ☰ menu

Section	Items
Commit	Copy artifacts directory (Linux path, and Windows path when your server has path mappings), View in browser.
Batch	Copy output directory, View in browser. Move runs to another batch, Rename batch.
Redo	Redo deleted runs, Redo failed runs, Redo all runs…. With a filter, they redo only the runs that match it. See Running from the web app.
Delete	Delete failed runs…, Delete all runs…, Delete multiple batches. With Delete files, keep metadata, only the files are deleted, optionally only those matching patterns like *.png.

Batches that are milestones are protected: see What a milestone keeps.

The messages above the views

• N running / N pending: hover to list the runs. Stop runs cancels them.
• N failed: up to 5 failed runs are listed: each name opens its log, and the target button next to it adds id:<run id> to the filter, to show only this run. When more failed, often for a common reason, the list is collapsed: List the N runs opens it. Show only the failed runs adds status:failed to the filter, Show all the runs removes it, and Open the logs shows the Logs view.
• N of the runs below were deleted: Redo deleted runs runs them again.
• Be careful, those are local runs: they were started from someone's workspace, and may not match the committed code.
• This commit's artifacts were deleted, are incomplete, or are being recreated: when and by whom, and what's missing, with buttons to bring them back. See If runs don't start.
• For batches started from the web app: Your batch is waiting for LSF, The batch you started from QA-Board failed... See Batches started from QA-Board.
• On the views that show runs, what all the runs share: see Reading configurations.

View options

The View options panel, on the right (4 in the picture above), changes what the views show, and in which order. Press o to show or hide it. It lists only what applies to the view you're on:

• Visualizations: a switch per visualization, including the ones hidden by default.
• Files to show: when runs have several files for a visualization, like the stages of a pipeline or the frames of a sequence, which one every card shows. See Pipeline stages and frames.
• Viewers: Compare images sets how every image viewer compares new and reference: side by side, with a wipe, one at a time, or their difference. Then the project's own switches, if it has some.
• Configuration: the settings all the runs shown share, Hide them in each run, and Shorten long settings with ….
• Sort runs: by name, batch, database, status, configuration, run id, storage, a tuning parameter or a metric, for the tables, the run cards and the logs. Until you choose, runs are sorted by the project's default metric, else by batch when they come from several, else by name, marked (default).

Reset goes back to the project's defaults, except for the sorting.

More about the panel

• Collapsed, it's a thin rail with an icon per section: hover an icon to see what the section is set to, click it to open the panel there. A dot on an icon means the section differs from the project's defaults.
• It starts open when the window has room for it and the results, else collapsed. Your choice is remembered.
• Resize it by dragging its left edge (the results make room when you let go), or focus the edge and use the arrow keys. Double-click the edge for the default width. On narrow windows, the open panel covers the page: Esc closes it.
• The number next to its title counts what differs from the project's defaults: hover it for the list.
• Visualizations: the header counts how many are shown. Visualizations without files in these runs are listed apart. A search box appears when there are many.
• Files to show: until you pick a value, each run shows its own default (its last value when the values are numbers, like its last frame, else its first). The value you pick applies to all runs; a run without it shows the closest one, with a note on its card. Per run lets you choose in each card instead. Changing an option keeps the image viewers' zoom and region.
• Compare images can also be set above any image, or with the keys 1 to 4.
• Sort runs: the order reads like the values: Highest first, A to Z, Newest first, Failed first... Each starts in its natural order: names A to Z, failed runs first, metrics highest first. Names are sorted "naturally" (frame2 before frame10), runs without a value come last, and ties are sorted by name. Batch and Database are listed when the runs come from several. It sorts by the new runs' values. The Summary doesn't use it.

The views

The sidebar lists the views, grouped by what they're about. Each one answers a different question.

View	Use it to...
Summary	Get the big picture: how the main metrics changed	Reading metrics
Metrics Table	Check each run's metrics against their targets	Reading metrics
Metrics Diff	Find the runs that improved or regressed vs the reference	Reading metrics
Visualizations	Look at images, videos, plots, text... next to the reference	Run cards, then the image, video and other viewers
Output Files	Browse and diff every file the runs created	Output Files
Logs	Find out why a run failed, follow running runs	Logs and failed runs
Artifacts & Configs	Diff the build and configuration files with the reference commit	Artifacts & Configs
Code	Open the commit on your git server	Code
Available Tests	See, check and edit the batches of tests you can run	Choosing the tests to run
Run Tests / Tuning	Run this commit's code on more tests, or with other parameters	Running from the web app
Tuning Analysis	See how the metrics depend on tuning parameters, and the best ones	Running from the web app

The sidebar also has the project's Integrations (see Running from the web app) and its milestones. Hover a view's name for a reminder of what it shows. Some projects open another view by default. The view you're on is part of the page's link: see Sharing.

Run cards

The Visualizations view has one card per run. A card starts with:

• the run's name, usually its input. Hover it for its batch, input path and database (click to copy), metadata, compute time, date and storage, and History: this input's results over time, in History. The logs and the tables show the same when you hover a run's name. When the runs shown come from several batches or databases, e.g. a batch that runs the same images with different settings, names start with them: night › kitti › scenes/0001.png (see below).
• tags for its platform and configuration, and how they differ from the reference run's: see Reading configurations. Orange Input, Config or Platform tags (with a ≠) mean the reference run is not exactly the same: see How runs are matched.
• its main metrics, each with its change against the reference run, colored when it gets better or worse: see what the colors mean.

Then come the visualizations the project configured, new next to reference. A file identical to the reference's is tagged Identical to the reference. Cards load their files as you scroll to them.

The facade's card: its PSNR, in red, still misses its 30 dB target, but its change, +5%, is green: better than the release. Then the regions saved for this photo, and both images, zoomed in together.

Next to the run's name, the ⋯ menu has the same as hovering its name, then redoes the run, or deletes it or only its output files, once you're signed in: see Redo, stop and clean up runs. It's the same menu in the logs.

Hover a run's name: History follows this input over the commits; below, its batch, its input, its metadata, and who sent it, its compute time and its storage.

The folder icon opens the run's output directory (or the reference's) in the file browser; the copy icon copies its path, as a Linux or a Windows path. The button at the card's top right shows the card in full screen.

To gather files from all the runs in one folder, use Export, next to the view's title: see Exporting files.

Compact cards, and the batch and database in runs' names

Compact cards: pick Compact next to the view's title (Comfortable brings the cards back as above). Each card's header is one line, like the runs in the logs: its name, its configuration as text (hover it for the tags, and how they differ from the reference), its compute time and its buttons. There is less room around the visualizations. It applies to the Visualizations and Output Files views, and is also in the View options' Display section and in Settings → Preferences.

Batch and database in names: by default, runs' names start with their batch and their database only when the runs shown come from several, e.g. night › frames/0001.png and day › frames/0001.png. Databases are shortened to the end of their path that tells them apart: hover them for the full path. When all the runs shown have the same input, names are only what differs, e.g. night and day: hover a name for its input. Filter the runs to one batch and the names are the inputs again. To always or never show them, use Batch & database in names in a run's menu or when hovering its name, the View options' Display section, or Settings → Preferences. It applies to the run cards, the logs and the tables.

Output Files

Every file each run created, as a tree, compared with the reference run's files. Next to the view's title, the storage the runs shown use.

Only the files that differ from the reference are listed: here, the pipeline stages that a rounding change touched. Click Help for what the icons mean.

• By default, only the files that are different, added or removed are listed. Filter them by path above the files: match:name lists identical files, diff:name different ones.
• File options, next to the filter, opens their options in the View options: Show all files, Hide runs without files, Expand all folders and Color-blind friendly. The button says which are on.
• Click a file to see its diff with the reference's. Ctrl-click or Shift-click to diff several. Next diff jumps to the next change.
• Help explains the icons and colors: bit-accurate, different, missing in the reference, only in the reference...

Click a file to diff it in place: Next diff jumps to the register that changed. Ctrl-click another file to diff it too. On a commit that only changed docs, the run is Bit-accurate.

When all the files your project checks are identical to the reference's, the run is tagged Bit-accurate. Files the project doesn't check are grayed and tagged not checked, like metrics.json, which has the run time. This is the view for bit-accuracy tests.

Artifacts & Configs

Compares the commit's artifacts with the reference commit's: the configuration files, the build, and anything else the project saves with qa save-artifacts. Pick which artifacts to explore with the buttons at the top.

Between two quantization commits, only build/flags.json changed: its diff shows which flags.

It works like Output Files: only the differences are listed, click a file to diff it. The buttons at the bottom copy the artifacts' path or open their folder. When a commit has no artifacts, the view says to run qa save-artifacts, usually in your CI.

Code

Opens the commit on your git server, in a new tab. Hover it for the commit's changes, its files (in a monorepo, the project's folder), the branch's files and commits, and the repository.

Where to go next

• Reading metrics
  Did the commit make things better, and where? The Summary, the tables and their colors.
• New and reference
  Pick what you compare with, and how runs are matched.
• The image viewer
  Find where two images differ, down to the pixel.