Skip to main content

2017

Where it all started: a small internal web page to browse and compare the results of an algorithm's CI runs, commit by commit, instead of digging through output folders.

Users

A results browser for CI runs

Each commit that ran in CI got its own page with its metrics, plots, videos and parameters, next to the list of recent commits. Results were read straight from the folders the CI wrote.

Users

Comparing two versions

Two commits could be compared side by side, with metric histograms and a summary table, to see at a glance whether a change improved or regressed the algorithm.

Web app​

  • Added a searchable list of commits. Each commit page shows the results on every input: metrics, curves, rendered videos and the parameters used.
  • Added an interactive view of the estimated trajectories, and support for ground truth to compute error metrics such as drift.
  • Commit pages show their git branch, and load faster thanks to caching.

Server and administration​

  • The app follows the CI's git-based output folders, and can be served with uWSGI behind nginx.

Notes​

  • QA-Board started in September 2017 at Samsung as a small internal Flask server, built to debug the results of a computer-vision (SLAM) pipeline faster. The ideas that shaped the product were already there: one page per commit, a reference to compare against, and rich visualizations of each output.