Skip to main content

Install the qa command line

qa is QA-Board's command line. It runs your code on your inputs, saves the files your code writes, and sends the metrics to your QA-Board server. Install it on every machine that runs your code: your computer, your CI runners, your cluster's nodes.

Install and connect​

  1. Install qa where your code's dependencies are, e.g. your project's virtual environment: qa imports your Python code to run it. It needs Python 3.7 or later.

    pip install qaboard

    If your code is an executable that qa only starts, you can install qa as a tool instead: uv tool install qaboard.

    caution

    The latest release on PyPI, 1.1.1, doesn't have some features these docs describe yet: qa gate, qa auth, Metric, QABOARD_TOKEN, commit statuses on GitHub. Until the next release, install qa from the repository to use them:

    pip install "qaboard @ git+https://github.com/Samsung/qaboard"

  2. Tell qa where your QA-Board server is, e.g. in your shell's profile:

    export QABOARD_URL=http://<your-qaboard>

    <your-qaboard> is your server's address, e.g. localhost:5151 for the quickstart's. No server yet? The quickstart starts one on your computer; for your team, see Starting a QA-Board server.

    Projects usually save shared results (qa --share, CI runs) in /mnt/qaboard (storage in qaboard.yaml), but the quickstart's server reads them in /tmp/qaboard: tell qa with export QA_STORAGE=/tmp/qaboard. Local runs don't need it: they save their results in your project's output/ folder.

  3. Check that qa runs. It lists its commands:

    qa --help
    Usage: qa [OPTIONS] COMMAND [ARGS]...
    [...]
    Commands:
      batch                        Run on all the inputs/tests/recordings in...
      check-bit-accuracy           Checks the bit accuracy of the results in...
      [...]
      init                         Provide a sample qaboard.yaml configuration.
      [...]
      run                          Runs over a given input/recording/test and...

Next, add QA-Board to your project with qa init.

Your organization's site package​

Your organization may publish a site package: a small Python package that configures qa for its server. It sets the server's address and other defaults, so you don't need the environment variables below. Install it next to qaboard, e.g.:

pip install --upgrade 'qaboard-site-mysite @ git+ssh://git@git.example.com/infra/qaboard#subdirectory=deployments/mysite/cli'

A site package can also give every project default qaboard.yaml settings, like where results are stored or the LSF queue: see site defaults.

tip

Environment variables always win over the site package's defaults: set QABOARD_URL to try another server without uninstalling anything.

Connect to your server​

qa finds your server's address in these environment variables:

VariableExample
QABOARD_URLhttps://qaboard.kitefin.exampleThe server's address.
QABOARD_HOST, QABOARD_PROTOCOLqaboard.kitefin.example:5151, httpsThe host (and port), and the protocol: http by default. They win over QABOARD_URL.
QABOARD_HOSTNAME, QABOARD_PORTqaboard.kitefin.example, 5151Both together win over the others.

Without any of them, qa uses http://localhost:5151 and prints a warning. With qa --offline, it never contacts the server: your runs are only saved on disk.

Outside CI, each qa command tells the server it ran: the command, your user name, the qaboard version, the project, the runner, how long it took and whether it failed, never your inputs or results. Admins see how QA-Board is used from it (usage analytics). It's sent in the background, and adds at most a second when the server is slow. Set QA_NO_ANALYTICS=1 to send nothing.

If your server uses HTTPS with a certificate your machines don't trust, e.g. from your company's certificate authority, point QABOARD_API_VERIFY to the CA bundle, or turn off the checks of qa's requests to the server:

export QABOARD_API_VERIFY=/etc/ssl/certs/internal-ca.pem
export QABOARD_API_VERIFY=false

Sign in​

Sign in once per server, so that the results you send are signed with your name, and so that qa can do what needs a signed-in user (e.g. start tuning or redo runs from scripts):

qa auth login
# Your one-time code: WDJB-MJHT
# To sign in, open https://<your-qaboard>/device and enter the code

Open the link in any browser, even on another computer: it works when qa runs over SSH. Sign in to QA-Board as usual, check the code and approve. qa saves a token for this server in ~/.config/qaboard/hosts.yaml, readable only by you (%APPDATA%\qaboard on Windows). It serves all your checkouts and worktrees: it's never saved in your projects.

qa auth status   # which server, which user, which token
qa auth logout   # signs out: revokes the token qa auth login created, and forgets it
qa auth token    # prints the token, e.g. for curl

The token expires after 180 days without use (admins can change it): run qa auth login again. You can see and revoke it in the web app, in Settings → CLI sign-ins.

In CI and scripts, give qa a token in QABOARD_TOKEN: create one in Settings → Access tokens (see CI integration). qa uses $QABOARD_TOKEN (or $QA_TOKEN), then the login of qa auth login, then QA_TOKEN in ~/.qaboard/secrets.yaml (the older way: it still works, for your default server only). A token you save with qa auth login --with-token isn't revoked by qa auth logout (add --revoke).

In CI, when the server refuses the results (an expired token, a token for other projects, a server that requires tokens...), qa run and qa batch fail before running anything, and say what to fix.

If qa doesn't start​

  • An error about your locale (ASCII, not UTF-8): set a UTF-8 one, e.g. export LC_ALL=C.UTF-8 LANG=C.UTF-8.
  • "Could not find a qaboard.yaml configuration file": run qa in a project, or create one with qa init.
  • "We are not sure where to find your QA-Board server": set QABOARD_URL, see above.
  • "The storage path does not exist": qa couldn't create the folder for shared results, storage in qaboard.yaml. Local runs only warn about it, but qa --share, CI runs and qa save-artifacts stop. Mount your server's results folder there, or point qa to it with QA_STORAGE, e.g. export QA_STORAGE=/tmp/qaboard for the quickstart's server. See Where is the data saved?