Send results without a shared storage
Runs can send their files to the QA-Board server over HTTP, instead of saving them in a folder that the server mounts too. Use it where a shared storage is hard to get: CI runners hosted by GitHub or GitLab, a laptop outside the office network, a cluster without the shared volume.
Sending results over HTTP is new. Tell us how it goes with your projects.
Set it up
-
Ask your server's administrators whether it accepts files. They turn it on with one setting. In a project,
qa doctorsays whether it does. -
Leave
storageout of qaboard.yaml, or say it.qa initwrites astorage: replace it.qaboard.yamlstorage: uploadFor one machine only, like a CI job in a project that otherwise uses a shared storage, set
export QA_STORAGE=upload. -
Sign in: files are always sent with a token. Run
qa auth login, and in CI let the job sign in as itself or give it a token.
Then use qa as usual: qa --share run, qa batch in CI, qa upload and
qaboard.start_run() all send their runs' files.
export QABOARD_URL=https://qaboard.example.com
qa --share run --input images/a.jpg
# ...
# Sent 12 files to QA-Board (3.4 MB)
What is sent, and when
A shared run is saved like a local one, in the project's output/ folder on the machine that runs it. qa sends
its files from there, and the web app shows the run like any other: same pages, same viewers, same links.
| When | |
|---|---|
| The run's log, and the series it logs | While it runs, every few seconds |
| Its metrics | While it runs for start_run(), at the end for qa run |
Files matching outputs.live in qaboard.yaml | While it runs, each time they change |
| All its other files | When it ends, before it shows as finished |
To look at some files before a long run ends, like plots it rewrites, list them:
outputs:
live:
- "plots/*.png"
- "*.html"
- A run that fails or is interrupted still sends what it wrote, with its log.
- A run that is done again replaces the files the server had.
- Files the server can't take, e.g. larger than its limit (1 GB by default):
qanames them, and the run still shows with its other files. In CI the command fails; elsewhere it prints the error and goes on. Save big outputs in a shared storage. - A server that doesn't accept files refuses the run when it starts: in CI,
qastops there. - When the server doesn't answer,
qatries again a few times, then says what is missing. The files stay in output/.
What still needs a shared storage
The server only has what runs sent it. It can't start your code where it has no build and no inputs.
| Without a shared storage | |
|---|---|
| See, compare, export and delete runs; the quality gate; History | Yes |
qa save-artifacts | Yes: it sends the commit's artifacts to the server |
| Start runs and tuning from the web app, redo runs | Yes, on the server's machine or on Kubernetes, from the artifacts qa save-artifacts sent (artifacts). Not with the LSF or Celery runners yet |
qa check-bit-accuracy | Yes: it compares the reference's manifests (its files' hashes), fetched from the server. Not with QA_BITACCURACY_CMP, which compares files |
qa optimize | Yes: it reads the iterations' metrics from the server. Its journal, to resume, stays on the machine that runs it |
qa batch with a cluster runner | When the cluster's nodes share the project's folder with the machine that starts the batch |
| Runs done again | In place: the server keeps the latest attempt's files |
A project can mix both: most machines save in the shared storage, and the ones that can't mount it set
QA_STORAGE=upload.
How it works
qacreates the run on the server, which chooses its folder under the server's own uploads folder.- Each file is a
PUT /api/v1/output/<id>/files/<path>, with its SHA-256: the server checks what it received before the file shows. The log is sent by what was added. See the HTTP API. - The run's files are all there before the run ends: from then on they don't change.
- A commit's artifacts are saved the same way, next to its runs, with
PUT /api/v1/commit/<commit>/artifacts/files/<path>: see the HTTP API. - Runs send the state of their folder, not events: after a failed request, the next one sends what is missing.
Where next
- Upload results from any toolTest reports and benchmarks from a CI job, without a run() function.
- AdminAccept results over HTTPTurn it on, size it, and keep it safe.
- Where is the data saved?With a shared storage.