Skip to main content

2020-q3

qaboard v0.8.13

QA-Board was announced publicly as an open-source project. Projects got a simpler API to write their run() function, separate storage locations for outputs and artifacts, and milestones are now one click away in the sidebar.

Users

QA-Board is open source

QA-Board was introduced to the public on August 10, with its source on GitHub, documentation and a blog, starting with posts introducing the project and showing how to create and compare flame graphs.

Learn more →
Project integration

Simpler run() API

Your run(context) function now reads everything from attributes like context.input_path, context.output_dir, context.params and context.forwarded_args, instead of digging into context.obj.

Learn more →
Project integration

Separate storage for outputs and artifacts

The storage key in qaboard.yaml, which replaces ci_root, can set different locations for run outputs and for build artifacts. Outputs are saved with new, shorter path conventions.

Learn more →
Users

Milestones in the sidebar

The project sidebar lists the project's shared and private milestones, so you can open one in a click. Commits that are milestones can no longer be deleted, and the cleanup of old results keeps them.

Learn more →

Web app​

  • Output cards show when each run started.
  • Metrics are formatted with 3 significant digits by default, or a per-metric precision, and use the metrics configuration of the latest commit.
  • The metrics-over-time plot shows all batches.
  • Added menus to delete all the batches of a commit.
  • Filtering on JSON values such as configurations is easier, and mismatch tags are clearer.
  • Integrations handle CI jobs that are blocked waiting for a manual action.
  • The web app no longer fails when main_metrics are missing.

CLI and project setup​

  • qa batch --action-on-pending chooses what to do when matching runs are already pending: wait, reuse their results, skip, or run anyway.
  • Input paths can be absolute, without specifying a database, and the default database is /. Database paths are normalized, and paths with many .. are handled better.
  • Long output directory names are shortened automatically.
  • QA_BATCHES_FILES lists the batches files of the current qa batch, and database paths can contain environment variables.
  • qa run warns when run() doesn't return a dict like {"is_failed": False}, and the sample project shows how to print logs in real time.
  • The CLI prints YAML parsing errors and shorter URLs.
  • GitLab CI statuses are updated more reliably and link directly to QA-Board for custom batches; tuning runs no longer update them.
  • qa get artifacts_branch gives where a branch's artifacts are saved.
  • Input manifests are limited to the first 200 files of an input folder.
  • Added a sample script that uses the QA-Board API to get a batch's total storage.
  • The qaboard package is now version 0.8.13.

Server and administration​

  • More backend settings can be set with environment variables.
  • Database backups can run as a different user.
  • nginx serves more common file types (YAML, shell and Python scripts) as plain text.
  • The backend handles commits with a large number of outputs better, and creates missing storage folders.

Documentation​

  • Published the introduction blog post and a flame graphs tutorial, with a Blog link in the navigation bar.
  • Added instructions to pull the docker images instead of building them, and docs on local runs and metrics.
  • The docs use the new Algolia DocSearch and better meta tags.

Fixes​

  • Fixed relative and absolute database paths, including with input types.
  • Fixed subproject URLs when the storage root contains the subproject.
  • Fixed crashes with undefined or invalid metrics, and a mixup between configurations and metrics.
  • Fixed forwarded arguments, dynamic outputs and tuning runs.