Skip to main content

2022-q1

qaboard v1.0.1

The qaboard package reached 1.0. The server now ships its own Celery task queue, string metrics show up in tables and cards, and you can delete only the output files that match a pattern.

Admins

Built-in Celery task queue

The QA-Board server now starts a RabbitMQ broker and a Flower monitoring dashboard, so projects only need to register Celery workers to run batches or tuning from the web app. Set runners.default: celery in qaboard.yaml to use it.

Learn more →
Project integration

String metrics

String metrics returned by your run() are now displayed in the metric tables and output cards, next to the numeric ones.

Learn more →
Users

Delete output files by pattern

When deleting a batch's files while keeping its metadata, you can now give patterns like *.png to delete only the matching files.

Project integration

Custom bit-accuracy comparisons

qa check-bit-accuracy can compare files with your own Python function, set with QA_BITACCURACY_CMP, instead of only comparing file hashes.

Learn more →

Web app​

  • Pages with many runs now render much faster.
  • Added a "Color-blind friendly" switch to the file and bit-accuracy viewers, and clearer file icons.
  • The image viewer now warns when an image file is empty.
  • Plotly figures saved as stand-alone .html files are now displayed, next to the reference's.
  • You can now delete a batch, or the output files of a milestone, even when another batch of the same commit is a milestone.
  • The tuning batch selector now shows how many runs are still running.
  • The sample tuning configuration mentions keep_all_best_iters.

CLI and project setup​

  • The qaboard package is now version 1.0.1.
  • Celery runners now get the environment variables of the qa batch command that started them (except ones that look like secrets), and default to the broker on your QA-Board server.
  • LSF job names are now short and safe to reuse as docker container names.
  • Auto-optimization: keep_all_best_iters can be set to true/false in the tuning configuration.
  • Output manifests are sorted by file name.
  • The CLI logs more details on network errors, and no longer fails when no tests are configured or when GitLab is not used.

Server and administration​

  • Added RabbitMQ and Flower services to the docker compose setup.
  • The database uses a custom PostgreSQL configuration file with more shared memory, and the backend README explains how to use your own.
  • The cleanup job now marks commits as deleted when their artifacts were deleted, and errors on corrupted git clones are now visible.
  • Auto-tuning jobs use an LSF queue with longer time limits.

Documentation​

Fixes​

  • Fixed a crash of the video viewer when the reference is bit-exact.
  • Fixed the image viewer's toolbar buttons with OpenSeadragon 3.
  • Fixed LDAP login, subproject support, and errors when opening commits that appeared twice.
  • Fixed storage usage being undercounted after deleting files by pattern.
  • Fixed auto-optimization running iterations more than once with parallel sampling, and failures with tuning pipelines.
  • Fixed errors when requesting too many metrics from the CLI (URLs above the maximum length).
  • Fixed metrics with labels in the commits evolution charts, and the width of the logs.