Tuning test resource usage

Bazel runs test actions in parallel, sized to the machine’s CPU and memory. A test that needs more than its share, such as one using a GPU or a large dataset, can run out of memory when others run beside it, and fail only sometimes. Tell Bazel what the test needs, and it schedules fewer actions around it. Start with the narrowest control that fixes the failures, and widen only if you need to.

Run a test alone: `exclusive`

The exclusive tag makes Bazel run the test with no other build or test action on the machine. Use it when a few specific tests consume far more than the rest.

BUILD.bazel

py_test(
    name = "gpu_stress_test",
    # ... other attributes
    tags = ["exclusive"],
)

Reserve CPU slots: `cpu:N`

The cpu:N tag reserves N CPU slots for the test. A higher N leaves fewer slots for other actions on the same machine, so it doubles as a memory reservation. Use it when exclusive is too restrictive and you want to tune how much concurrency to give up.

BUILD.bazel

py_test(
    name = "medium_gpu_test",
    # ... other attributes
    tags = ["cpu:4"],
)

Declare a custom resource: `resources:<name>:<amount>`

For a resource Bazel doesn’t count, such as GPUs, declare how many the machine has and how many each test uses. Bazel then never runs more tests at once than the machine can hold.

BUILD.bazel

py_test(
    name = "gpu_test",
    # ... other attributes
    tags = ["resources:gpu:1"],
)

.bazelrc

test --local_resources=gpu=2

Limit concurrency for the whole build

These flags affect every action, and are usually more than a few problematic tests need. Consider them when a large part of your test suite is resource-intensive.

Flag	Effect	Use it when
`--test_strategy=exclusive`	Runs every test alone	Nearly all tests need the machine to themselves
`--local_test_jobs=N`	Caps how many tests run locally at once	Tests are the constraint, and build actions can keep their parallelism
`--jobs=N`	Caps how many actions run in parallel across the build	Machine capacity is the constraint for everything

​Run a test alone: exclusive

​Reserve CPU slots: cpu:N

​Declare a custom resource: resources:<name>:<amount>

​Limit concurrency for the whole build

Run a test alone: `exclusive`

Reserve CPU slots: `cpu:N`

Declare a custom resource: `resources:<name>:<amount>`

Limit concurrency for the whole build