Skip to main content

Adding QA-Board to your project

Go at the root of your project's git repository and run:

qa init
#=> 🎉🎉🎉

Along with previously existing files and directories, your root directory will now contain a structure similar to:

root-git-repository
├── qaboard.yaml       # configuration
└── qa
   ├── main.py         # sample entrypoint that runs your code
   ├── batches.yaml    # examples of how to run multiple tests
   └── metrics.yaml    # examples of how to define KPIs

Site defaults​

Organizations can provide defaults for all projects (where results are stored, which task runner to use...) with a base qaboard.yaml:

  • Each project's qaboard.yaml is merged on top of it, and can override any setting. Mappings are merged key by key, lists are replaced unless they contain super (e.g. batches: [super, qa/batches.yaml]), and locations (storage, inputs.database) are replaced as a whole.
  • qa init doesn't copy the sample project's values for settings the site defines, it shows the site's values as comments instead.
  • To check the configuration a project ends up with, run qa get config.

The base configuration is found via the QABOARD_SITE_CONFIG environment variable, which site packages set (e.g. a qaboard-site-mysite package points to its own qaboard_site_mysite/qaboard.yaml). Read the installation docs to install your site package. Set QABOARD_SITE_CONFIG="" to ignore it.

If you set QABOARD_SITE_CONFIG yourself, e.g. as a CI variable shared by Linux and Windows runners, it can depend on the platform like other locations:

QABOARD_SITE_CONFIG='{"linux": "/mnt/qaboard/site.yaml", "windows": "//server/qaboard/site.yaml"}'
Creating a site package

Add a qaboard.yaml next to your site package's __init__.py, and add to its defaults:

"QABOARD_SITE_CONFIG": str(Path(__file__).with_name("qaboard.yaml")),

(Optional) Connect your git host​

QA-Board works with GitHub (and GitHub Enterprise), GitLab, Gitea/Forgejo and Bitbucket Cloud. When your repository sends its push events to QA-Board, you can:

  • See your commits in QA-Board with their message, author and avatar, find them by branch or tag
  • Get direct links to your code
  • Have qa batch set a commit status that links to the results (GitHub and GitLab)
  • Start and follow CI jobs from the web application: GitHub Actions workflows and GitLab CI manual jobs
  • Wait for CI pipelines to end when checking if results changed vs the last version
  • Delete old results

Your QA-Board server must know your git host, and have a token if the repository is private: ask its admins, or see Git hosts. If a delivery fails, its HTTP status tells why (see webhooks).

GitHub​

In the repository's Settings > Webhooks, add a webhook with:

  • Payload URL: https://<your-qaboard-server>/webhook/github
  • Content type: application/json
  • Secret: the server's webhook secret, if it has one (recommended)
  • Events: Just the push event

GitHub sends a "ping" right away: in Recent Deliveries you should get a 200 response 🎉.

GitLab​

As one of the project's Maintainers, go to Settings > Webhooks (https://<gitlab>/<group>/<project>/-/hooks) and add a webhook with:

  • URL: https://<your-qaboard-server>/webhook/gitlab
  • Secret token: the server's webhook secret, if it has one (recommended)
  • Trigger: Push events and Tag push events

To test everything went well, GitLab lets you "Test" your new hook. You should get a blue happy 200 OK message 🔵🎉.

Gitea, Forgejo and Bitbucket​

Like GitHub, with the payload URL https://<your-qaboard-server>/webhook/gitea (or /webhook/forgejo) and the Push event, or for Bitbucket Cloud https://<your-qaboard-server>/webhook/bitbucket with the Repository push trigger.

tip

project.url in your qaboard.yaml (e.g. git@github.com:org/repo.git) tells qa and QA-Board where your repository is hosted, even before the first push event.