Key concepts

QA-Board shows the results of your code, commit by commit, next to a reference. This page explains the words it uses, in the order you meet them.

The command line and the server

qa is the command line. It runs your code where you start it: your computer, a CI runner, a cluster. It wraps your code with a small Python function, saves each run's output files and sends its metrics to the server. See Running your code.
The server stores what runs report in a database, and serves the web app. It reads the output files from a storage that every machine running qa also mounts, at the same path (by default /mnt/qaboard, often an NFS share). See Where is the data saved?
qaboard.yaml, at the root of your project, tells qa how to run your code, where your inputs are, which metrics matter and how to show your outputs. See Project setup.

Project: usually a git repository, sometimes a folder of one. project.name in qaboard.yaml is its id, e.g. kitefin/camera-isp.
Commit: a version of your code. QA-Board shows results per commit: the commit page is where you spend most of your time.
Input: what a run works on: an image, a video, a recording, a load profile, a folder. Inputs are paths, relative to a folder set in qaboard.yaml (the database). See Inputs.
Configuration: the settings of a run, e.g. base or low-light. Their meaning is up to your code. Tuning adds parameters on top, shown as key: value tags. See Configurations.
Platform: where the code runs or what it targets, e.g. linux or a hardware model. Runs are compared on the same platform.
Run: your code from one commit, on one input, with one configuration, on one platform. A run has output files, metrics and a log.
Batch: a group of runs of the same commit, with a label. qa batch ci runs the inputs and configurations listed under ci in your batches file. The batch CI runs has the label default, and the web app shows it as CI. Tuning experiments and your own runs get their own batches. See Batches of inputs.

Metrics: numbers each run returns: quality, speed, errors. metrics.yaml gives each one a label, a unit, a target and whether smaller is better, so the web app can color improvements and regressions. See Metrics.
Outputs: the files runs write, shown side by side with the reference's: images with synced zoom, plots, flame graphs, text, videos, HTML reports. qaboard.yaml says which files to show and how. See Visualizations.
New and reference: a commit's results are always shown next to a reference's. By default the reference is the latest commit with results on the project's reference branch (project.reference_branch). Each run is compared with the reference run of the same input, with the closest configuration.
Milestone: a commit and batch kept as a reference: a release, a benchmark, "before my refactor". Shared milestones are for everyone in the project, private ones for you. See References and milestones.
History: a metric across the commits of a branch, run by run, to find the commit that changed it. See History.
Tuning: batches that try parameter values, started from the web app or with qa optimize, then compared in the Analysis view. See Tuning from the web app.