Skip to main content

2026-10

qaboard v1.1.1

QA-Board now works with GitHub as well as GitLab, Gitea/Forgejo and Bitbucket, searches all of a project's commits, opens big commits much faster, and has a file browser and a video viewer. A quickstart, a user guide and release notes in the app help you get started; admins, check your git host settings before upgrading, and at SIRC install one more package when you upgrade qaboard.

Project integration

GitHub and other git hosts

QA-Board works with repositories on GitHub and GitHub Enterprise as well as GitLab, and receives the push events of Gitea, Forgejo and Bitbucket Cloud. On GitHub, qa batch sets a commit status that links to the results, and you can start GitHub Actions workflows from the web app. The experimental qa gate fails the CI when quality regresses.

Learn more →
Users

Search all your commits, browse your files

The commit list's Filter box searches all of a project's commits, not only the last days: by message, committer, branch, batch or hash. Open a folder under /s/ to get a file browser. Commit pages with 1000 runs filter and sort without freezing, and big commit lists and the projects list load much faster.

Learn more →
Users

What's new, a user guide, logs and videos

Release notes pop up once a month is over: read them again from the sparkles button in the sidebar. A new user guide, with screenshots of demo projects, explains the web app. Logs update live, with search, failed LSF runs show why LSF stopped them, and videos compare with a wipe, frame by frame.

Learn more →
Admins

Quickstart, Kubernetes, and upgrading

Try QA-Board with one Compose file and demo projects. deployments/deploy.py rolls out new backend replicas next to the old ones and rolls back instantly, and each release publishes its images and a Helm chart. Before upgrading, keep GITLAB_HOST set or list your GitLab in QABOARD_GIT_HOSTS: a migration records the git host of each project.

Learn more →

Web app​

  • Added the "What's new" panel: the release notes, newest first, with a search box. The link /#whats-new opens it, handy to share in a chat.
  • The "What's new" dialog pops up once a month (or quarter) is over, with everything that changed in it. The notes of the current month show in the panel as a preview, tagged "In progress".
  • The sidebar's help button opens the new user guide.
  • The Logs view lets you filter runs (all, failed, running), and tags failed runs with the reason (e.g. "LSF: out of memory").
  • In the Logs view, each run fits on one line, with its configuration in short and buttons to open its output directory, redo, mark as failed or delete it. The full configuration shows when you open the run's logs, and the logs no longer get wider than the window.
  • The sidebar's Logs entry shows how many runs failed, or are still running.
  • Switch between a run's log.txt, log.lsf.txt and log.dask.txt. For LSF runs, a summary shows the exit reason, host, run time and memory used vs requested, with hints that point to the batch's lsf: options.
  • Batches started from QA-Board (tuning, "Run Tests") show whether qa batch is waiting for LSF, running, failed (with the error) or didn't start any run, with a link to its logs.
  • Follow links to code, commits and branches on your project's own git host, with the right URLs for GitHub, GitLab, Gitea/Forgejo and Bitbucket, instead of a hardcoded GitLab server.
  • Shows the "Quotas" links only when the server has a quota dashboard.
  • The error page links to your server's support page.
  • The URL now keeps everything you select: the commit list's date range, the metrics shown in tables, the commits highlighted on the History page. Share the link, reload, or use the back button: you get the same view. Typing filters and toggling viewer options no longer adds entries to the browser's history.
  • Dynamic options (e.g. which frame to show) work across runs with different values: the controls panel lists the values of all runs, and a run that doesn't have the selected value shows the closest one and says so. Options are linked across runs from the start, and unlinking one gives each card its own selector, starting from the same value. See the user guide.
  • Links are much shorter: the URL only keeps what differs from the defaults (batch, filters, metrics, view, visualization controls, plot options).
  • Open a folder under /s/ in your browser to get the new file browser: sort and filter files (e.g. *.png), copy their Linux or Windows paths, paste a path to go there, and navigate with the keyboard (press ?). Scripts and curl still get nginx's plain index.
  • In the file browser, a bar shows the run whose folder you are in, with its status and a link to its results: redo it, delete it or mark it as failed from there.
  • The Available Tests view is a real editor for your private and shared batches: it flags YAML mistakes as you type, shows if your changes are saved next to the Save button (or Ctrl+S), lets you review or discard them, and keeps them if you leave before saving. It also warns instead of overwriting someone else's changes to the shared batches. See the user guide.
  • The Available Tests view lists the batches in each file: go to one, list the tests it resolves to on the commit, or run it in one click.
  • The Available Tests editor flags inputs that qa batch can't use as you type: an empty -, or a number, date or yes that should be in quotes.
  • When a batch has no tests in Available Tests or Run Tests / Tuning, see why: a file of inputs.batches that isn't in the commit's artifacts (with the path QA-Board looked for), a file that isn't valid YAML, a batch that no file defines, or inputs the server didn't find under the database. Available Tests also lists the batch files it read, and which one defines the batch. See the user guide.
  • Copy the path of a run's output directory from the Logs view, next to the button that opens it. When your server has Windows path mappings, the confirmation offers the Windows path too. The batch directory, and the folders of batches started from QA-Board, have the same buttons.
  • See where the logs of each command that started a batch are: in the CI job (with a link), or in QA-Board (one click shows them). The batch's own log.txt only shows when it exists. See the user guide.
  • Show more of very long logs: they show their last 50,000 lines to stay fast, and say if the hidden lines have errors or matches for your search.
  • When you filter a batch's runs, the commit bar menu redoes only the runs that match the filter. Follow redone runs in the Logs view, under "Started from QA-Board". See the user guide.
  • Find your way around from the History page: the sidebar shows the project's navigation (the branch's commits, Code, History, Integrations, Milestones), and moving between these pages no longer reloads the app.
  • See who made a commit at a glance: when there's no avatar picture, committers, users and projects show their initials on a color that stays the same everywhere.
  • The Analysis view of a qa optimize experiment plots its convergence itself: each iteration's objective and the best one so far, even when matplotlib isn't installed where qa optimize ran.
  • Run cards fit the window when qaboard.yaml doesn't set their width (outputs.style.width): they used to be 1500 px wide, wider than a laptop's screen.
  • Link to History with or without the breakdown per test, or relative to the first commit: ?breakdown_per_test=false and ?relative=true in the URL now work.
  • Compare videos with a wipe, by toggling between them in place, or as their difference, besides side by side. Zoom with the mouse wheel or a pinch and drag to pan: both videos follow, and zoomed in you see the actual pixels. See comparing videos.
  • Step through videos frame by frame with the arrow keys, scrub, slow down to 0.1× or loop: new and reference stay on the same frame, and wait for each other when one is still downloading.
  • Copy the current frame of a video (new, reference or their difference) to the clipboard, or save it as a PNG.
  • Start the GitHub Actions workflows declared in your project's qaboard.yaml from the Integrations menu, and follow the run you started, even when GitHub ran it on a newer commit of the branch. Only users who can access the project can see or start them (GitHub Actions).
  • See the status of a GitHub Actions workflow's latest run for the commit, with a link to it, in the Integrations menu.
  • See your git host's defaults in the Integrations menu of branch pages, when your project configures none and QA-Board knows its git host: GitLab's pipeline status, or a link to GitHub Actions (default integrations).
  • Hides the Integrations menu's badges that fail to load, e.g. for private projects or projects without CI.
  • See commit authors' avatars from GitHub, Gitea/Forgejo and Bitbucket, learned from push events, or else their Gravatar.
  • Loads avatars and badges from public git hosts (github.com, Bitbucket, Codeberg, and your server's git hosts that aren't GitLab) directly, which is faster. Those from your GitLab hosts and Gravatar still load through the server, which can reach hosts you can't.
  • Search all of a project's commits from the commit list's Filter... box, not only those in the date range: by message, committer, branch, batch or hash. Use branch:, committer:, batch:, message: or id: to look at one field, and -word to exclude (searching commits).
  • Scroll through long commit lists: they show the 50 latest commits, then more as you scroll, so big projects open much faster. Branch pages still get their whole date range at once (up to 1000 commits), for their metrics chart.
  • Click Older commits at the end of the commit list to look a week further back.
  • Open the projects list much faster on sites with hundreds of projects: it downloads less, and shows more projects as you scroll.
  • Jump to a branch from the Filter... box: when your search can be a branch name (one word, or a branch: term), it suggests the most recently active branches that match, even in projects with thousands of branches.

CLI and project setup​

  • At SIRC, upgrading qaboard now requires also installing qaboard-site-sirc, see the installation instructions. Site packages provide a base qaboard.yaml that every project's configuration is merged on (site defaults): qa init shows the inherited values instead of overriding them.
  • LSF runs write their output live to log.lsf.txt, with LSF's job report at the end. qa batch no longer passes an LSF queue or project when none is set, and fails fast when bsub is missing.
  • qa init creates a configuration that runs locally: LSF is commented out, and storage defaults to /mnt/qaboard like the server.
  • The CLI reads your QA_TOKEN from $QA_TOKEN or ~/.qaboard/secrets.yaml, and warns if other users can read that file.
  • QABOARD_API_VERIFY controls the TLS verification of the CLI's requests to the server: true (the default), false, or the path to a CA bundle.
  • Site packages (company-specific plugins) are no longer ignored on Python 3.8 and 3.9.
  • The package now includes the qatools compatibility package, and update checks no longer suggest a site-specific install command to everyone.
  • Read the logs of qa --share batch and qa optimize started from a terminal in QA-Board, with the batch's commands: they save what they print in the batch directory (under qa-batch-logs/), errors included. qa optimize includes the output of its iterations. In the CI, the CI keeps the logs. See the CLI tips.
  • Configure the Windows runner's PowerShell $ErrorActionPreference per job (the batch's error_action: option) or with the QA_RUNNER_ERROR_ACTION environment variable (default Stop).
  • Tune how long qa batch waits for the Windows runner's Jenkins builds: 2 hours by default (previously 15 minutes), polling every 30 seconds (previously 5) with the QA_JENKINS_TIMEOUT and QA_JENKINS_STATUS_INTERVAL environment variables.
  • qa batch --runs-file runs.json does a list of runs, each with its input, configurations, parameters and runner options. QA-Board uses it to redo runs together. See the CLI tips.
  • With --no-wait and an existing Dask cluster (scheduler_address), the Dask runner now enforces concurrency: a driver LSF job sends the runs, as without scheduler_address. See the Dask runner.
  • Fail the CI when quality regresses, with the experimental qa gate: set rules in qaboard.yaml (gate:) on the median of a metric vs a reference branch, milestone or commit, on each test, on failed runs or on bit-accuracy. It prints a table (or Markdown, or JSON), fills the GitHub Actions job summary, and can post a commit status to GitHub or GitLab that links to the Metrics Diff. See Quality gate.
  • Batches list their inputs in the same order on every machine: the inputs found by globs are sorted by path, instead of coming in the order the filesystem lists them.
  • Set the frame rate of a video visualization with fps in qaboard.yaml, when the web app's guess is wrong. See the video viewer.
  • qa batch sets a commit status that links to the results on GitHub too (CI integration).
  • qa check-bit-accuracy waits for the GitHub checks of the reference commit too, like it does for GitLab pipelines (bit-accuracy).
  • In GitHub Actions, qa records the pull request's branch and last commit instead of the temporary merge commit (CI integration).
  • In GitHub Actions, qa records the tag of tag builds.
  • qa finds your git host, its API and its token in QABOARD_GIT_HOSTS too (git hosts).
  • qa sends GITHUB_ACCESS_TOKEN only to github.com and the GitHub hosts you configured, not to any hostname that looks like GitHub.

Server and administration​

  • Zero-downtime deploys with deployments/deploy.py: images tagged per version, several backend replicas behind nginx, rolling updates, rollback, status and restore-db.
  • deploy.py seed-db fills a staging database with only production's recent data.
  • A Helm chart to run QA-Board on Kubernetes.
  • The backend has a /api/v1/health endpoint, shuts down gracefully, and keeps working when redis is down.
  • Security: actions that run commands (redo, tuning, stopping or deleting batches...) now require a login and run as the logged-in user, and their inputs are escaped. See SECURITY.md.
  • Security: the server only writes, deletes or runs code in the folders listed in QABOARD_STORAGE_ROOTS (default: /mnt/qaboard), and refuses results stored elsewhere. If your projects store results in other folders, set it before upgrading (see environment variables).
  • Require a secret in push webhooks: set QABOARD_WEBHOOK_SECRET, and the same secret in your repositories' webhooks, so that only your git hosts can send push events (webhooks).
  • Each batch started from QA-Board gets its own folder with its scripts and logs, under the batch's qaboard-batches/.
  • The web app links to the docs served by your server at /docs/, which match its version, instead of the public website. Set QABOARD_DOCS_ROOT to use another docs site.
  • Configure the quota dashboard and support links with QABOARD_QUOTA_URL_TEMPLATE and QABOARD_SUPPORT_URL.
  • Security: the server only loads avatars and badges for the web app from your GitLab hosts and gravatar. If your projects' avatar_url or badges (integrations[].src) come from other hosts, list them in QABOARD_PROXY_HOSTS (see environment variables).
  • Restrict who can read the files of restricted projects, or of the folders you list under paths in users_restrict.yml: nginx asks the backend before sending each file (see user management). If you use your own nginx configuration, include snippets/qaboard-files.conf in the /s/ locations to get the file browser.
  • In the folders listed in QABOARD_FILES_UNIX_PERMISSIONS (e.g. /home), users only read the files that their own Unix account can read (see user management).
  • Without a complete /etc/passwd, e.g. with LDAP or SSSD, QABOARD_ACCOUNTS_COMMAND (and QABOARD_GROUPS_COMMAND) gives the backend the users' accounts, refreshed daily: the file browser shows owners' names instead of their ids (see users' accounts).
  • Redoing, deleting or marking a run as failed now requires access to its project, for restricted projects.
  • Scripts can send results that describe their commit with only commit_sha and commit_authored_datetime: the other commit_* fields (branch, message, committer, parents) are optional, and the server doesn't need access to the git repository.
  • Add pages for your own users to the docs your server serves: put them in website/site-docs/<site>/ and build with QABOARD_SITE=<site>. They get their own menu, and the installation page links to them. At SIRC, "At SIRC" has the install commands and SIRC specifics.
  • Pull QA-Board's images from ghcr.io/samsung/qaboard: docker-compose.yml uses them by default, each release has its own tags (e.g. backend-1.2.0), and QABOARD_IMAGE picks another registry.
  • Install QA-Board on Kubernetes from the Helm chart published with each release, oci://ghcr.io/samsung/charts/qaboard (see Kubernetes).
  • Try QA-Board on your laptop with kind: charts/qaboard/quickstart.yaml and charts/qaboard/kind.yaml start a small cluster where the qa CLI can save results (see the quickstart).
  • Mount the storage where runs save results with the chart's new results setting (an existing claim, a new volume, NFS or a node folder): the chart mounts it where the backend, nginx and Cantaloupe expect it, and sets QABOARD_STORAGE_ROOTS.
  • The website image (the docs your server serves) is built from the repository's root, like the frontend image: if you build it without docker compose, run docker build -f website/Dockerfile ..
  • At SIRC, QA-Board no longer serves the storage over WebDAV (/davs/algo/): it let anyone write and delete files. services/nginx/webdav.conf explains how to bring it back.
  • Get the numbers of the Summary and Metrics Diff views from scripts and agents, with the experimental /api/v1/commit/<sha>/compare: each metric's median vs a reference, each run vs its reference run, and how many runs failed. See Quality gate.
  • Try QA-Board on your computer with demo projects to look at: download one Compose file, deployments/quickstart/compose.yml, and run docker compose up -d --wait (see the quickstart). It only answers on your computer.
  • Show the docs' demo projects on a Helm install with the chart's new demoData setting, which the kind quickstart (quickstart.yaml) turns on: they're restored once, before QA-Board first starts, and never over existing data. The demo data is published with each release, as ghcr.io/samsung/qaboard:demo-data-<version>.
  • Build the website image behind a proxy that blocks some of the docs' media (e.g. videos), or all of them: with QABOARD_MEDIA_MISSING=remote, readers' browsers load those shots from the media host or your mirror (QABOARD_MEDIA_BASE_URL), and with QABOARD_MEDIA_MISSING=placeholder they render as placeholders, instead of failing the build. A proxy's "blocked" page is reported as such, without retrying.
  • Configure all your git hosts (GitHub and GitHub Enterprise, GitLab, Gitea/Forgejo, Bitbucket, other git servers), each with its own token and webhook secret, with QABOARD_GIT_HOSTS (git hosts).
  • Keeps your current git settings working: GITLAB_HOST (also without a scheme, or with credentials), GITLAB_ACCESS_TOKEN, GITHUB_ACCESS_TOKEN, QABOARD_GITHUB_HOSTS for GitHub Enterprise, and QABOARD_GITLAB_HOSTS, whose hostnames are other names of GITLAB_HOST. A GitLab and a GitHub host can share a hostname (backward compatibility).
  • Records each existing project's git host, with a migration that uses the deploy's git host settings. Before upgrading, keep GITLAB_HOST set, or list your GitLab in QABOARD_GIT_HOSTS with the hostnames older projects used (upgrading).
  • Receives the push events of repositories on Gitea, Forgejo and Bitbucket Cloud, at /webhook/gitea (or /webhook/forgejo) and /webhook/bitbucket (webhooks, project setup).
  • Stops saving git tokens in new clones' .git/config, or showing them in error messages: git gets them from a credential helper, and only for their own host (it needs git 2.31 or newer). Clones made by older versions still have the token in their remote URL: remove it.
  • Stops sending git tokens, and GitLab session cookies, to another host when git or API calls are redirected there.
  • Reads git tokens and webhook secrets only from its environment, not from the CLI's shared secrets file (QA_SECRETS).
  • Refuses push events for git hosts without a webhook secret, once any host has one or QABOARD_WEBHOOK_SECRET is set (401): otherwise anyone could claim a repository is on such a host.
  • Refuses push events from git hosts the server doesn't know (403), without reading the rest of the payload. Pushes to /webhook/gitlab from unknown hostnames still go to your GitLab, like before.
  • Refuses push events that would move an existing project to another git host (409).
  • Refuses webhook payloads over 25 MB (413), the most GitHub sends.
  • Answers and ignores webhook events other than pushes, like GitHub's ping.
  • Keeps starting, and running migrations, when the git host settings are invalid: the server logs the error, and refuses webhooks (503) until you fix them.
  • Lists of commits answer many times faster on big projects, and use much less memory: they no longer load every output of every commit.
  • API: page through /api/v1/commits with limit and offset (the X-Has-More response header tells if there are more), and search all of a project's commits with q (lists of commits). Without them, it answers as before.
  • API: get the most recently active branches matching a text with /api/v1/project/branches?q=...&limit=....
  • API: get a much smaller list of projects with /api/v1/projects?summary=true.
  • Adds two indexes on ci_commits, to list commits faster. They are built without blocking writes, which can take a few minutes on big databases. If building one fails, the next deploy builds it again.
  • Starts even when the database has migrations it doesn't know, made by a newer version, e.g. after a rollback: it keeps them.
  • Versions from before October 2026 don't start on a database migrated by this one: to roll back to them, first undo the newer migrations (rollbacks).
  • The Helm chart's migrations Job gets the chart's config: settings, which data migrations need (Kubernetes).

Documentation​

  • New user guide for people who use the web app every day.
  • New runbooks for deployments, rollbacks and Kubernetes.
  • Release notes going back to 2017, on the website and in the app.
  • Many pages updated to match the code, and features that were never documented: GitHub webhooks, API tokens, batch options, bit-accuracy manifests, auto-optimization...
  • Find the server docs in one place, written for any site: starting a server, deployments and rollbacks with deploy.py, host maintenance, backups and troubleshooting.
  • Follow old links to pages that moved or were removed: they redirect to the closest page.
  • Find auto-optimization in the sidebar.
  • Write your own file viewer: see the end of visualizations.
  • See where QA-Board is going on the new roadmap: its principles, and for each theme what's shipped, experimental, next or still being explored.
  • See each image viewer tool at work in screenshots and short clips of a demo project, in Comparing results and The commit page.
  • See the web app in screenshots and clips of the demo projects on every page of the user guide, and next to the settings that produce them in the project integration docs, like Visualizations.
  • Run QA-Board on your computer with the new quickstart: one Docker Compose file, or a kind cluster and the Helm chart, with demo projects to look at, then results from an example project.
  • Put your project on QA-Board step by step with the tutorial: pick your server, code hosting, CI and runners once at the top, and every step shows the instructions for your stack. Links keep your choices (e.g. ?server=k8s&ci=gitlab-ci).
  • See what QA-Board is for in QA-Board in 5 minutes and in one use case per demo project of a fictional robotics company, starting with image quality. The key concepts fit in one page.
  • Find the quickstart, the review loop and what's coming next on the website's new home page.
  • Links to the docs shared in chats and social networks show a card with QA-Board's tagline and a glimpse of the web app.
  • The docs show a placeholder that describes a screenshot or clip when your browser can't load it, instead of a broken image.
  • New git hosts page for admins.
  • The project setup and CI pages cover GitHub and GitLab equally (CI integration).

Performance​

Measured on a commit with 1000 runs:

  • Type in the filter box of a commit page without the page freezing: the results update in half a second, instead of after 7 seconds of a frozen page. Sorting runs, and matching them with their reference, no longer freeze the page either.
  • Open the Metrics Table in a third of a second instead of 7 seconds, with a fraction of the memory (40 MB instead of 640 MB): big tables only render the rows on screen.
  • Open the Visualizations twice as fast: cards download their files as they scroll into view, and files shared by several cards or viewers are downloaded once. Batches with more than 300 runs show more cards as you scroll, or all of them with "show all".
  • Commit pages load in 0.6 s instead of 1 s.
  • Come back to a page you already visited and it shows up at once, then refreshes in the background. Commits with running outputs keep updating.
  • After a reload, the projects and your account show at once, from your browser's cache, while QA-Board fetches fresh data.
  • Open tuning batches faster: runs that have an exact match in the reference no longer go through a name-similarity search.
  • Scroll through hundreds of runs in the Logs view without slowdowns.
  • Open very long logs (tens of thousands of lines) several times faster, and search them without the page freezing for seconds.
  • See the diff of big text files (logs, configurations...) with the reference sooner: the editor loads while the files download, instead of after.

Fixes​

Across the web app​

  • Copying the path of an output, batch or commit directory gives the right path when folder names contain characters like #, and the confirmation shows the copied path.
  • Notifications show at the bottom of the window again, instead of at the bottom of the page's first screen where they were out of sight once you scrolled down.
  • Counters say "1 output", "1 run" and "1 project" instead of "1 outputs", "1 runs" and "1 projects".
  • Links to the LSF docs and to the getting-started page work again, and the retired chat link now points to your server's support page (by default, QA-Board's GitHub issues).
  • The "No projects yet" page links to your server's docs and support page.

Commit page and viewers​

  • Dynamic options no longer show an empty card when a run doesn't have the selected value, sort numbers as numbers (2 before 10), and no longer split into options like frame_1234 that control nothing. Whether an option is linked is no longer remembered across projects.
  • In the image viewer, changing a Dynamic Option (e.g. a slider over your pipeline's stages) keeps the zoom and the selected ROI, for the new and reference images, instead of zooming out to the whole image, most often when the reference's image showed up. Images with another aspect ratio fit the selected ROI again.
  • Hiding a visualization shown by default takes one click instead of two.
  • A run is compared with the reference run of the same platform: when the reference batch had the same input and configuration on two platforms, the other platform's run was picked.
  • The Improvement report showed "=" for metrics that changed, and badges for those that didn't.
  • The Summary's change in % has the right sign for metrics with negative values, and runs that return no value (null) for a metric no longer count as 0 in its median.
  • The Summary only warns that it mixes runs with different parameters when the runs it aggregates really have different parameters: filter the runs to compare one configuration.
  • In the Metrics Table, the New and Ref headers stay over their columns when a filter leaves out every run that has some metric, and metrics without a target show just their unit, like [s].
  • Removing a metric from a table's metric picker no longer also drops the selected metrics that the current batch doesn't have.
  • The reference commit's refresh button works again: it was disabled once the commit had loaded.
  • The error page shows the error as text, instead of inserting it in the page as HTML.
  • Output Files tags runs Bit-accurate when their output files match the reference's: like qa check-bit-accuracy, it only checks the files that match bit_accuracy.patterns in qaboard.yaml (all files by default, except logs, run.json and metrics.json). The other files are still listed when they differ, greyed and tagged "not checked". See bit-accuracy tests.
  • Images show in the image viewer on servers that don't set QABOARD_IMAGE_SERVERS: the default is now Cantaloupe's /iiif/2/ endpoint, instead of a /iiif URL that Cantaloupe couldn't answer.
  • JPEG images show in the image viewer again, instead of an "Image Download Error": the image server only sent them as JPEG tiles, and the viewer asks for PNG ones. They now come as PNG tiles, with the exact pixel values for the pixel tooltip and the diff. Update the cantaloupe image to get the fix.
  • The image viewer's Auto ROIs finds regions again, instead of failing with an error.
  • The image viewer's ROI thumbnails show the image the viewer shows: they no longer break for runs that save a TIFF instead of a PNG, or for images served by another image server (QABOARD_IMAGE_SERVERS).
  • ROIs set on inputs of another size than the output images are placed right on non-square images, in the thumbnails and when you zoom on them.
  • In the image viewer, selecting a region (the selection button, or c) draws it again and shows its histograms, instead of panning the image.
  • The image viewer's diff threshold slider no longer jumps to its maximum and gets stuck: it moves step by step after you turn the diff on.
  • In point clouds, WASD moves while the key is pressed, not after it's released.
  • Plots saved with side_by_side: false draw the reference run's traces on the same axes, instead of nothing when a figure has several traces. Traces keep their own dash style.
  • Differential flame graphs are colored by what changed: red for frames that take more time than in the reference, blue for less, orange for new ones. The colors no longer exaggerate changes near the root.
  • The SLAM and tuning plots no longer have Plotly's "Send to Cloud" button, so their data can't be uploaded to Plotly's public cloud.
  • Text viewers set to show only differences hide identical files, and videos play without a reference.
  • Text files in a run's output files show again when the reference run doesn't have them, and their diff with the reference no longer breaks when you press t to swap the sides.
  • HTML outputs (like a run's report) show their images and links, which are relative to their own folder, and their styles no longer change the rest of the page: they show in a frame, without running scripts.
  • Artifacts & Configurations shows the artifacts again when the commit loads after the view, instead of "Could not find" them.
  • Playing or pausing a video that has no reference no longer raises errors ("pause is not a function").
  • Videos your browser can't play (e.g. HEVC) say so, with the command to convert them, instead of staying black. Animated .webp and .gif files now show, side by side with the reference's, instead of a black box or their bytes as text.
  • The Summary's box plots on a log scale show a few round ticks, instead of labels that overlapped.
  • In the video viewer's Toggle mode, clicking "Toggle" again flips between the new and reference videos, like the t shortcut.

Logs​

  • The Logs view's copy buttons (logs, commands) work when QA-Board is served over plain http, not only https.
  • Switching between a run's log files shows the end of the new file, instead of keeping the scroll position and the highlighted error of the previous one.
  • The floating panel's button no longer hides the buttons on the right of the Logs view.

Commits, branches and history​

  • Reloading a page, opening a shared link or using the back button shows what the URL says: the selected commits, batches and filters could disagree with it.
  • The back button takes you back in one click after following links inside QA-Board: some links added two entries to the browser's history.
  • The chart above a branch's commits shows the branch's metrics, the median or average over each commit's runs, instead of staying empty. Each commit row also shows the median and average of the default metric over its CI runs.
  • History's "Performance over time" chart shows your project's metrics again, instead of asking you to define them.
  • In History, the chart's axis no longer shows the metric's unit when the values are relative to the first commit (@100).
  • The History page's per-test tables show their metric columns again, in tabs named "Metrics Table" and "Metrics Diff" like on commit pages.
  • On the History page, picking metrics updates the metrics distribution too.
  • Selecting a milestone from qaboard.yaml selects the latest commit of its branch.
  • The batch selector no longer jumps back to the top while you scroll it, and its keyboard navigation works again. It now opens on the selected batch, marked with a check.
  • Switching to a branch you had opened in a previous visit no longer crashes the commits list ("setHours is not a function").
  • Commit pages and the branch list no longer break when results were sent without a branch.
  • The commit list's day headers say "1 commit" instead of "1 Commits".
  • The projects list shows the description from qaboard.yaml (project.description) for projects that aren't on GitHub or GitLab too.
  • History's y axis no longer cuts off the start of its top label ("000 ms" for 1000 ms).
  • The chart above a branch's commits leaves out the commits that don't have the metric, instead of drawing gaps in its line.
  • Typing a commit id in the commits list's search and pressing Enter no longer leaves "[object Object]" in the search box.

Runs, batches and tuning​

  • Redoing runs from QA-Board respects the batch's concurrency, and uses the runner the runs were started with (e.g. Dask) and its settings. QA-Board used to start each run with its own qa batch, so they all ran at once, with the default runner.
  • Runs that start again (redo, qa batch...) no longer show as failed, and count as failed, until they finish: their status is reset when they are pending or running.
  • Finished runs whose metrics don't include is_failed count as successful, instead of neither successful nor failed.
  • A stuck run's "Mark as Finished" action is now labelled "Mark as Failed", which is what it does.
  • Moving runs to another batch closes the dialog, and saving a milestone no longer risks losing it.
  • The tuning form shows a readable error instead of the server's raw response.
  • The tuning form remembers the "Overwrite" option, and setting the number of iterations no longer crashes.
  • Run Tests / Tuning runs as you when you open it from a link: it no longer asks for a user it can't let you type, with Send disabled.
  • The Auto-Tuning Analysis plots each iteration's metrics (those in the objective) next to the objective, for tunings run from now on: qa optimize used to get no metrics from the server.
  • Projects whose inputs are inside the project (a relative inputs.database, like inputs) see their tests in the Run Tests / Tuning form and in Available Tests: they are looked for in the commit's artifacts, where the runs started from QA-Board run, instead of in a folder of the server. In monorepos, from the root qaboard.yaml's folder, like qa does.
  • Run Tests / Tuning and Available Tests find the inputs of projects whose inputs.database uses {project} or {subproject} (e.g. /datasets/{project.name}), instead of looking for them from /.
  • When several of a commit's batch files define the same batch, Run Tests / Tuning and Available Tests use the last one, like qa batch, instead of the first.
  • Listing a batch's tests no longer fails for commits that QA-Board saw before qa sent their qaboard.yaml (e.g. opened by branch name): it uses the project's, and the commit gets its own the next time qa runs on it.
  • Run Tests / Tuning lists the tests of batches whose name contains +, & or #.
  • When a batch has an input it can't use, like an empty - or a number, qa batch says which batch, in which file and at which line, instead of "Could not understand the inputs in the batch" followed by all its inputs. Available Tests and Run Tests / Tuning show the same error.
  • Runs started from QA-Board are the logged-in user's, instead of the server's user.
  • Stopping a batch whose runs can't be stopped reports the error instead of failing.

File browser​

  • Going back after opening a file shows the folder again, instead of its raw listing.
  • Links to a run's output directory open the file browser even when the folder doesn't exist, e.g. for runs that failed before they started, instead of an error page.

CLI​

  • With LSF job arrays and the Dask runner, qa run no longer deletes the log file its output goes to: errors are no longer lost.
  • Runs started from a git worktree checkout are saved under their commit, instead of their branch name. Submodules with a relative .git pointer also work.
  • On Windows, qa batch finds existing results for projects whose name contains a "/", instead of failing with "could not find any data on commit ID": --action-on-existing works there.
  • qa optimize works with recent versions of scikit-optimize, instead of failing with "cannot import name 'Integer'".
  • qa optimize no longer fails at the end when matplotlib isn't installed: it only skips its convergence plot.
  • qa batch --tuning-search runs a grid search when the JSON doesn't say search_type, as the docs show, instead of failing with KeyError: 'search_type'.
  • qa no longer warns that it doesn't know where your server is when you set QABOARD_URL=http://localhost:5151.
  • qa no longer prints a git error when the commit it runs on isn't in the local clone (e.g. shallow CI clones): it only did without the commit's details. QA_VERBOSE=1 still shows it.
  • With Python 3.12 and later, qa no longer prints a SyntaxWarning: invalid escape sequence when it starts.
  • qa batch accepts a single input written as text (inputs: a.jpg), and says that a batch written as a list (my-batch: [a, b]) must be a mapping with inputs:, also in the web app (the server used to call exit() there).
  • qa get works with Python 3.13: it said "Could not find" for every variable.
  • QA_BATCH_CONCURRENCY no longer overrides the concurrency of the runs that qa batch --runs-file redoes, as if --local-concurrency was given.
  • bit_accuracy.on_reference_failed_ci: compare-first-parent goes back up to bit_accuracy.max_parents_depth parents, instead of two at most.
  • bit_accuracy.failed_ci_job_name works again: qa check-bit-accuracy only looks at that job (<job> <subproject> in subprojects).
  • qa check-bit-accuracy counts skipped CI jobs as successful, instead of waiting for them.

Server and administration​

  • The project's settings (metrics, visualizations, storage...) are no longer replaced by those of whichever branch got new results last: they come from the reference branch (project.reference_branch in qaboard.yaml), or from the project's first results.
  • Upgrading the Helm chart no longer restarts the backend for nothing after a first install (its settings checksum included a generated secret that rendered differently each time).
  • nginx starts on hosts without IPv6, instead of failing on its listen [::]:80 line.
  • Repositories whose clone is broken are cloned again, instead of failing until an admin deletes them. The broken clone is only replaced once the new clone succeeded.
  • Tag pushes record the tag's whole name: its first character was cut.
  • Pushes to repositories with files or folders whose name only looks like qaboard.yaml (for example an agent skill folder named qaboard-yaml) no longer fail: only files named exactly qaboard.yaml or qatools.yaml define projects.
  • Jenkins integrations show the status of the builds they trigger when Jenkins' own URL uses another hostname than the integration's build_url (for example a full domain name): checking them failed with a 403 because no credentials matched that hostname in JENKINS_AUTH.
  • Servers outside SIRC no longer show avatars from SIRC's directory for GitLab users without one: set QABOARD_AVATAR_URL to use your own (environment variables).

Security and data safety​

  • Marking a run as failed or deleting it now requires a login.
  • Deleting results no longer deletes the contents of folders that they link to.
  • When a restricted project saves its results in a folder per user (e.g. /algo/outputs/{user}), the project's files are now restricted in every user's folder, not only in the server's.
  • The API qa uses to update batches no longer lets anyone without a login delete runs. qa optimize still replaces its previous best iteration's runs, and only those.
  • When qa fails to update QA-Board, its warning no longer prints your API token (QA_TOKEN).
  • When signing up with a user name or email that is already used, the server answers "already taken" instead of an error that included the password's hash, and no longer writes the password to its logs.
  • Logging in with an empty password is refused, and LDAP logins escape the user name.
  • The server only deletes the git clones it made: it refuses repository paths inside a clone, or that hold other clones.