A quieter quarter: commits whose artifacts were cleaned up can be restored from the web app, output cards can go full screen, and failed runs can be redone in one click.
When old artifacts of a commit were deleted to save space, its page now offers to restore them, so that its deleted outputs can be redone.
Learn more →Each output card has a button to view its files full screen.
QA-Board was announced publicly as an open-source project. Projects got a simpler API to write their run() function, separate storage locations for outputs and artifacts, and milestones are now one click away in the sidebar.
QA-Board was introduced to the public on August 10, with its source on GitHub, documentation and a blog, starting with posts introducing the project and showing how to create and compare flame graphs.
Learn more →Your run(context) function now reads everything from attributes like context.input_path, context.output_dir, context.params and context.forwarded_args, instead of digging into context.obj.
Learn more →The storage key in qaboard.yaml, which replaces ci_root, can set different locations for run outputs and for build artifacts. Outputs are saved with new, shorter path conventions.
Learn more →The project sidebar lists the project's shared and private milestones, so you can open one in a click. Commits that are milestones can no longer be deleted, and the cleanup of old results keeps them.
Learn more →QA-Board got ready for outside users: the server now starts with docker compose, projects no longer need GitLab, and the CLI works in GitHub Actions. New flame graph viewer, matrix batches, and many image viewer improvements.
The server is now a set of single-purpose images (backend, web app, database, image server, reverse proxy) started with docker compose, with optional daily database backups and a simple restore script.
Learn more →Outputs saved as *.flame.json are displayed as interactive flame graphs, with differential flame graphs to compare against the reference and pinpoint performance regressions.
Learn more →The CLI now sends the commit, branch and configuration data to the server itself, so QA-Board works without GitLab and without giving the server read access to your repository. The CLI also integrates with GitHub Actions.
Learn more →Batches can define a matrix of configurations and platforms, and qa batch runs every combination.
Learn more →The project took its public name: qatools became QA-Board, with a `qaboard` Python package and a `qaboard.yaml` configuration, and its source was prepared for open-sourcing. Projects also gained Celery runners, monorepo subprojects, and new menus to rename, redo or clean up batches.
The configuration file is now qaboard.yaml and the package is imported with import qaboard. Existing qatools.yaml files and import qatools keep working, and the sample project now keeps its files in a qa/ folder.
Learn more →Besides local runs and LSF, batches can now be sent to Celery workers. Runners now share a common API, which makes it easier to add new ones, and local batches are the default.
Learn more →qaboard.yaml no longer needs to sit at the root of the repository, and nested qaboard.yaml files (even empty ones) define subprojects with their own settings.
Learn more →Commit pages can show integrations that start GitLab CI or Jenkins jobs, and batch logs link directly to the CI job that ran them.
Learn more →The web app got its name, QA-Board, and its documentation website. Bit-accuracy checks, milestones and batches defined in YAML arrived, and the image viewer became a real comparison tool.
qa check-bit-accuracy compares a commit's outputs with any reference (branch, tag or commit), with clear reports and file manifests to make it fast. In the web app, a bit-accuracy tab shows which outputs changed.
Learn more →Star important commits or batches as milestones, with a name and a description, and compare any result against them. Milestones can be kept for yourself or shared with the whole project.
Learn more →Images are compared with the reference with synced zoom, pixel diffs, color tooltips, histograms and brightness, contrast and gamma controls. New text, diff and HTML viewers cover the other outputs.
Learn more →Batches files list the inputs to run on, together with their configurations and database, and support aliases, filters and custom input discovery. Run them with qa batch --batch, on LSF or locally in parallel.
Learn more →The year QA-Board became a platform: a React web app backed by a database and a REST API, a `qa` command-line tool that wraps any project's code, tuning and automated optimization on the cluster, and the first generic viewers for images, plots and videos.
The server-rendered pages were replaced by a React single-page app, backed by a database and a REST API. It now hosts many projects, each with its commit list, per-commit results, filters and metrics over time.
A small Python package wraps your code so that each run on an input is tracked: qa run for one input, qa batch for groups of inputs on the cluster, and qa init to set up a project with a single YAML file.
Learn more →Parameter sweeps can be started from the web app and run on the cluster, then explored with dedicated plots. An optimizer based on scikit-optimize can also search the parameter space automatically to minimize a loss you define.
Learn more →Output files are shown with generic viewers: Plotly charts, videos, 3D point clouds, and an image viewer with deep zoom, side-by-side comparison and perceptual diffs against the reference.
Learn more →Where it all started: a small internal web page to browse and compare the results of an algorithm's CI runs, commit by commit, instead of digging through output folders.
Each commit that ran in CI got its own page with its metrics, plots, videos and parameters, next to the list of recent commits. Results were read straight from the folders the CI wrote.
Two commits could be compared side by side, with metric histograms and a summary table, to see at a glance whether a change improved or regressed the algorithm.