Skip to main content

Local Multiprocessing

By default, if you don't configure an async task queue, QA-Board will use joblib and local multiprocessing to run your batches.

Running batches locally​

If you configured an another runner, you can always:

qa batch --runner=local my-batch

Configuration​

The main option is concurrency (joblib's n_jobs). If it is not set, runs are done one at a time; the sample qaboard.yaml created by qa init uses -1 (1 run per CPU). You have multiple options to set it up:

  • Per batch:
qa/batches.yaml
my-batch:
  inputs:
  - image.jpg
  local:
    concurrency: -1 # 1 concurrent run per CPU
  • Globally:
qaboard.yaml
runners:
  default: local  # default
  local:
    concurrency: 1 # serially
  • On the CLI:
qa batch --local-concurrency 1
  • via the QA_BATCH_CONCURRENCY environment variable

You can also set a timeout (in seconds, 0 means no timeout) with runners.local.timeout, qa batch --local-timeout or QA_BATCH_TIMEOUT.

tip

Read joblib's docs