QA-Board

Algorithm engineering is hard enough.
Don't waste time with logistics.

[object Object]

What is QA-Board?

QA-Board helps Algorithms and QA engineers build great products. It offers powerful quality evaluation and collaboration tools.

[object Object]

What does it do?

Compare results between commits. Create advanced visualizations from your existing output files. Track metrics across time. Start tuning experiments.

[object Object]

How do I use it?

Run your code with a small CLI wrapper. You will see results from the web application.

Annotations

Organize & Share

Triggered runs from the CI or locally, and see them all in one place.

QA-Board is aware of version control; you can filter by commit, branch, author, message...

Always Compare

Whether you look at metrics or visualizations, QA-Board always compares each output to a reference version. You can save Milestones to benchmark new results.

You can compare configurations and filter results however you like.

Always Compare
Aggregation and rich KPIs

Rich Metrics

Real-world project need to look at heterogenuous KPIs: performance, quality, training time...

QA-Board lets you define as many metrics as needed, add metada (targets, label, units..), and can give you aggregated summaries as well as granular tables.

Output Visualizations

With QA-Board, each run dumps files in an output folder. When comparing results, you see a diff of all the files. Files are displayed with a wide range of viewers: first-party support for plotly, flame graphs, text, images, synced videos, 3d pointclouds, raw HTML...

You can declaratively create visualizations to show e.g. multiple images, debug data, sliders....

File-based Visualizations
Advanced Image Viewer

Advanced Image Viewer

QA-Board notably supports a performance image viewer based on OpenSeaDragon. At Samsung it lets use work smoothly with >100MP images.

Advanced features include showing perceptual differences, automatically finding interesting (or defining ahead of time) "Regions of Interest", use histograms, color tooltips and image filters.

Tuning & Optimization

Define batches of inputs to run on files/databases that matter to you. Start tuning experiments to compare parameters or feature flags

Use Grid-Search or Black-box optimization (via scikit-optimize), and analyse trade-offs. Use common tools for distributed runs.

Tuning & Optimization
Integrations

Integrations

From QA-Board itsef you can link to your docs, or artifacts. You can also add buttons to trigger 3rd party tools like Jenkins, GitlabCi, or webhooks. When runing during CI runs, QA-Board will update your CI tool with the run's status.

Regression Explorer

If a regression occured, you can simply quickly investigate when it happened, and diff the change.

More generally, QA-Board can be used for dashboard that show progress over time.

Regression Explorer

More features...

Mono-repo support, Bit-accuracy checks, Input metadata....