Starting a QA-Board server

QA-Board's run-wrapper, qa, will sync with a central server that tracks and displays results.

Try it first

The quickstart starts QA-Board on your computer with one Compose file, and demo projects to look at.

The quickstart: start one Compose file, and once every service is healthy, QA-Board answers on localhost:5151 with the demo projects.

Need a hosted version?

We're considering offering a hosted solution to help you get started. If you are interested, contact the maintainers.

Until then, please create an issue if you run into problems starting a server. We're responsive.

Create a directory to store results

QA-Board expects that all clients can access a shared storage to save and read results.

To get started quickly on a single server, create a local folder. Worry about sharing it later:

mkdir -p /mnt/qaboard
chmod -R 777 /mnt/qaboard

The base docker-compose.yml doesn't mount any results storage. Mount it in your site overlay (start from deployments/example.yml): the backend needs it at the same path as the clients, and nginx serves files under /s/ from /var/qaboard/ (/s/mnt/qaboard/... is read from /var/qaboard/mnt/qaboard/...):

deployments/mysite.yml
services:
  backend:
    volumes:
    - /mnt/qaboard:/mnt/qaboard
  proxy:
    volumes:
    - /mnt/qaboard:/var/qaboard/mnt/qaboard:ro

tip

If you want to use a different folder, use your path in the overlay, and set storage in your projects' qaboard.yaml (see where is the data saved).

When users open a folder under /s/ in their browser, they get QA-Board's file browser. nginx still sends the files, but the backend lists the folders: it lists those inside QABOARD_STORAGE_ROOTS, at the same paths as in the URLs. Other clients (curl, wget...) still get nginx's index. To restrict who can read the files of some projects, see user management.

Shared Storage?

Later, read how to setup NFS or Samba. If you need fine-tuning read about options for NFS volumes in docker-compose.yml.

Working in the cloud?

Use file-base storage like AWS EFS or GCP Filestore.

We plan on supporting blob-stores like AWS S3. Contact us or create an issue if it would help.

Starting the server

Prerequisites

You need Linux, docker, docker compose and git.

Deployment

QA-Board uses compose overlays in the deployments/ directory to adapt the stack to different environments, and an env file per environment that lists them:

git clone https://github.com/Samsung/qaboard.git
cd qaboard

Add your overlay to COMPOSE_FILE in a copy of deployments/production.env (the examples use the file as it is), then:

# first time
deployments/deploy.py deployments/production.env up --build --init
# upgrades (git pull first), with zero downtime
deployments/deploy.py deployments/production.env up --build

If you prefer plain docker compose, docker compose -f docker-compose.yml -f production.yml up -d still works, without zero-downtime upgrades.

A site overlay adds your site's services, volumes, environment variables and nginx configuration without modifying the base docker-compose.yml. Start from deployments/example.yml for the overlay, and from deployments/production.env for the env file.

deployments/deploy.py runs several backend replicas and replaces them one version at a time, runs database migrations once, and keeps the previous images for rollbacks. Read Deployments, rollbacks and high availability for the details.

tip

The application is live at localhost:5151 by default (set QABOARD_PORT_HTTP to change it).

Kubernetes

A helm chart is available in charts/qaboard, see Kubernetes.

Managing access rights and users

QA-Board can connect to external authentication systems (LDAP, SAML). If needed, access can be restricted to logged-in users. To know more about this, or create a test user, refer to the docs on user management.

(Optional) Environment variables

To configure your installation, set services.backend.environment in your site overlay (see deployments/example.yml). An .env file only sets the variables used by the compose files: the ports (QABOARD_PORT_*), and those that docker-compose.yml or your overlay forward to the backend (e.g. SECRET_KEY, GITLAB_ACCESS_TOKEN, QABOARD_DB_HOST, QABOARD_LOGIN_*). Others, like the LDAP settings, must be listed in the overlay.

ENV Variable	Default	Usage
`SECRET_KEY`	generated	Signs the sessions. If unset, a key is generated once and saved in the backend's data volume. With several hosts/kubernetes, set the same value everywhere.
`QABOARD_GIT_HOSTS`	none	Your git hosts (GitHub, GitLab, Gitea/Forgejo, Bitbucket...) with their tokens and webhook secrets: a JSON list, or the path to a JSON/YAML file. See Git hosts
`GITLAB_HOST`	https://gitlab.com, if `QABOARD_GIT_HOSTS` isn't set either	Your GitLab, e.g. https://gitlab.example.com. It's the default git host: for projects that don't say where they are hosted, and for pushes to `/webhook/gitlab` from hostnames the server doesn't know. Keep it set when you upgrade, see Git hosts
`GITLAB_ACCESS_TOKEN`	none	Token for `GITLAB_HOST`: `read_repository` scope to clone private repositories, `read_api` to show GitLab CI job statuses and committers' avatars, `api` to play manual jobs
`GITHUB_ACCESS_TOKEN`	none	Token for github.com (always known, even without a token) and the hosts of `QABOARD_GITHUB_HOSTS`: to clone private repositories, look up committers' avatars and use GitHub Actions (see GitHub integration)
`QABOARD_GITLAB_HOSTS`	none	Comma-separated. Hostnames (e.g. `gitlab-alias`) are other names of `GITLAB_HOST`. URLs (e.g. `https://gitlab2.example.com`) are other GitLab servers, that get `GITLAB_ACCESS_TOKEN`. `QABOARD_GIT_HOSTS` lets you give each its own token
`QABOARD_GITHUB_HOSTS`	none	Comma-separated hostnames of GitHub Enterprise servers, that get `GITHUB_ACCESS_TOKEN`. `QABOARD_GIT_HOSTS` lets you give each its own token
`QABOARD_PROXY_HOSTS`	none	Comma-separated hostnames the web app may load avatars and badges from through the backend, besides your git hosts (`QABOARD_GIT_HOSTS`, `GITLAB_HOST`...), the host of `QABOARD_AVATAR_URL`, and gravatar. Add the hosts of your projects' `project.avatar_url` and `integrations[].src` images
`QABOARD_STORAGE_ROOTS`	/mnt/qaboard	Folders where your projects store outputs and artifacts, separated by `:` (e.g. `/mnt/qaboard:/mnt/datasets`). The backend refuses results stored elsewhere, and never writes, deletes or runs code outside these folders
`QABOARD_FILES_UNIX_PERMISSIONS`	none	Folders, separated by `:` (e.g. `/home`), where users only read the files under `/s/` that their Unix account can read. See user management
`QABOARD_ACCOUNTS_COMMAND`	none	Command printing the users' accounts like `/etc/passwd` (e.g. `ssh host getent passwd`), for file owners and Unix permissions, when you can't mount a complete `/etc/passwd`. See users' accounts
`QABOARD_GROUPS_COMMAND`	none	Same for the groups, like `/etc/group`
`QABOARD_ACCOUNTS_REFRESH`	86400	How often (seconds) the backend runs these commands
`QABOARD_WEBHOOK_SECRET`	none	Secret that push webhooks must send (GitLab's Secret token, GitHub's Secret...), unless their host has its own `webhook_secret` in `QABOARD_GIT_HOSTS`. Recommended: without it anyone who can reach the server can send fake push events
`QABOARD_PORT_HTTP`	5151	Port mapped to the app on the host
`QABOARD_DB_HOST`	db	Connect the backend to a non-default database host (e.g. instead of dev'ing with prod dumps, connect directly to it)
`QABOARD_DB_PORT`	5432	Connect to a non-default database port
`JENKINS_AUTH`	none	Credentials used to trigger jenkins jobs on 1 or many jenkins servers. The format is a JSON string looking like `{"hostname_1": {"user": "jenkinsuser", "token": "xxxxx", "crumb": "yyy"}}` (how-to-get-the-token-crumb?)
`QABOARD_LOGIN_TYPE`	LOCAL	Set to `LOCAL/LDAP/SAML`
`QABOARD_LOGIN_REQUIRED`	empty	Any non-empty value (e.g. `true`) blocks anonymous users. Leave it unset/empty to allow them: even `false` enables it
`QABOARD_DISABLE_SIGNUP`	empty	Set to `True` (exactly) to disallow local users from signing up
`QABOARD_LOGIN_RESTRICTED`	empty	Any non-empty value (e.g. `true`) uses a configuration file to allow only specific users (requires backend restart)
`QABOARD_LOGIN_RESTRICTED_YAML`	none	The path to the users configuration file
`QABOARD_LDAP_HOST`	none	Server hostname (including port)
`QABOARD_LDAP_PORT`	389	Server port, usually 389 (or 636 if SSL is used / not supported yet!).
`QABOARD_LDAP_USER_BASE`	none	Search base for users.
`QABOARD_LDAP_BIND_DN`	none	The Distinguished Name to bind as, this user will be used to lookup information about other users.
`QABOARD_LDAP_PASSWORD`	none	The password to bind with for the lookup user.
`QABOARD_LDAP_USER_FILTER`	none	User lookup filter, the placeholder `{login}` will be replaced by the user supplied login. (e.g. `(&(objectClass=inetOrgPerson)(
`QABOARD_LDAP_ATTRIBUTE_EMAIL`	mail
`QABOARD_LDAP_ATTRIBUTE_COMMON_NAME`	cn
`QABOARD_SAML_DIR`	none	The path to the directory with the SAML configuration files, as in the python-saml docs
`QABOARD_SAML_ATTRIBUTE_EMAIL`	none
`QABOARD_SAML_ATTRIBUTE_COMMON_NAME`	none
`QABOARD_SAML_ATTRIBUTE_USER_NAME`	none
`QABOARD_SAML_ATTRIBUTE_ID`	none
`QABOARD_TUNING_RUNNER`	local	How tuning/"redo" runs started from the webapp are dispatched: `local`, `celery` or `lsf`
`QA_RUNNERS_LSF_BRIDGE`	none	With `lsf`: the command used to submit LSF jobs, e.g. `ssh user@lsf-host 'bsub_su {user} -I {bsub_command}'`
`QABOARD_DASK_SCHEDULERS`	none	Comma-separated dask scheduler addresses the backend trusts (to stop dask jobs from the webapp)
`QABOARD_IMAGE_SERVERS`	`{"default": "/iiif/2/"}`	JSON map of file extensions to IIIF image servers, e.g. `{"default": "/iiif/2/", "raw,dng": "https://iip.example.com/fcgi-bin/iipsrv.fcgi?IIIF=/"}`. Each URL is followed directly by the image's path
`QABOARD_PATH_MAPPINGS`	`[]`	JSON list of `["windows_prefix", "linux_prefix"]` pairs, used to show Windows paths to results (`qa` reads it too, from the environment or a site package, to convert Windows paths), e.g. `[["\\\\server\\share", "/mnt/share"]]`
`QABOARD_DEFAULT_USER`	qaboard	The user CI runs are saved under (with `{user}` in `storage`). Runs started from the webapp replace it with the logged-in user in output paths
`QABOARD_DOCS_ROOT`	/	Where the webapp's links to the docs point. By default, the docs served by this server at `/docs/`, which match its version
`QABOARD_AVATAR_URL`	none	Avatar URL template with a `{user_name}` placeholder, e.g. from a company directory: for the logged-in user's menu, and for GitLab committers who didn't upload an avatar. For the user's menu it's only read from the environment, not from a site package
`POSTHOG_API_KEY` / `POSTHOG_HOST`	none	Enables PostHog product analytics in the webapp
`REDIS_HOST` / `REDIS_PORT`	redis / 6379 (set in docker-compose.yml)	Redis cache shared by the backend replicas. The backend still works if it's unreachable
`QABOARD_QUOTA_URL_TEMPLATE`	none	Shows a "Quota" link, with `{user_name}` and `{project}` placeholders
`QABOARD_SUPPORT_URL`	GitHub issues	Where users report bugs, can be a `mailto:`
`CANTALOUPE_MEM_START`	1g	Starting memory for the image server (set in `services.cantaloupe.environment`)
`CANTALOUPE_MEM_MAX`	2g	Max memory for the image server (set in `services.cantaloupe.environment`)
`QABOARD_BACKEND_REPLICAS`	1 (2 in `deployments/production.env`)	Backend replicas, see Deployments
`QABOARD_UWSGI_PROCESSES`	16	uwsgi processes per backend replica, in production
`SENTRY_DSN`	none	monitor crashes in the backend with sentry.io. Example: https://xxxxxxxxxxx@sentry.io/000
`SENTRY_SAMPLE_RATE`	0.2	sample for perf monitoring

note

In the future we plan to introduce a proper "secret" store, per user and per project.

Consult the Troubleshooting page for examples that show how to get logs from the various services composing QA-Board.

For development, consult the READMEs for the backend and the frontend.

(Optional) For "production"

Backups

The cron-backup-db service in production.yml makes daily backups of the database. Mount a host directory at /backups in your site overlay, and copy the backups off the host. To restore one: deployments/deploy.py <env file> restore-db <file>.dump. See Backups.

Using SSL / hosting behind a reverse proxy

What we do is use our own copy of services/nginx/conf.d/qaboard.conf in the site overlay (keep its include snippets/... lines: they serve the files and the file browser). Files mounted under /tmp/deployment/nginx/conf.d/ in the proxy container are added to (or replace) services/nginx/conf.d/:

deployments/nginx/qaboard.conf (copied from services/nginx/conf.d/qaboard.conf)
server {
  # ...
  listen 443 ssl;
  ssl_certificate_key /ssl/cert.key;
  ssl_certificate /ssl/cert.pem;
  # ...
}

And mount keys with:

deployments/mysite.yml
services:
  proxy:
    ports:
    - "443:443"
    volumes:
    - "./deployments/nginx/qaboard.conf:/tmp/deployment/nginx/conf.d/qaboard.conf:ro"
    - "somewhere/cert.key:/ssl/cert.key"
    - "somewhere/cert.pem:/ssl/cert.pem"

Getting a certificate from your IT

Generate a key and a certificate signing request (CSR), and send the CSR to your IT:

openssl genrsa -out cert.key 2048
openssl req -new -sha256 -key cert.key -out cert.csr

If they send back a .cer certificate in the DER format, convert it to PEM: openssl x509 -in cert.cer -inform der -outform pem -out cert.pem.

If you want to use your own reverse proxy, with nginx for instance you can set QABOARD_PORT_HTTP=8080 and:

server {
  listen 80;
  server_name default_server;
  location / {
    proxy_pass http://localhost:8080/;
  }
}

(Optional) Cleanup

We run those cron jobs:

# Weekly cleanup of old results
# https://samsung.github.io/qaboard/docs/storage/deleting-old-data
59 1 1 * * cd qaboard && docker compose --env-file deployments/production.env exec -T backend qaboard_clean

# Weekly removal of old docker images, helps to avoid filling the disk on the host
59 1 2 * * docker image prune --force

tip

Check qaboard_clean --help to implement complex cleanup strategies.

Create a directory to store results​

Starting the server​

Prerequisites​

Deployment​

Managing access rights and users​

(Optional) Environment variables​

(Optional) For "production"​

Backups​

Using SSL / hosting behind a reverse proxy​

(Optional) Cleanup​