Interaction with Package Managers

Goals: Learn how Bazel interacts with Package Managers, including pinning dependency versions, Supply-chain security with resource integrity hashes, and configuring the Downloader To work with the code in the example repository, we need the toolchains and external dependencies that it relies on. Typically that means third-party dependencies, but the same techniques are useful for dependencies written by your company but managed outside of your Bazel workspace. By the end of this section, you should be able to run bazel fetch to download these for the language you pick.

bzlmod: Bazel’s package manager

DefinitionA “Bazel module” is a Starlark project that can have multiple versions, each of which publishes metadata about other modules that it depends on.

Recall that “Starlark” is a dialect of Python used to configure Bazel, as well as some other tools. Note that a .bzl file is called a “Starlark module” which is a different concept, representing the target of a load statement. Introduced in Bazel 6.0, “bzlmod” is the package manager for Bazel modules. It is semantically similar to numerous other dependency management systems, though extensive discussion went in to making it sufficiently generic. Read more in its documentation: https://bazel.build/build/bzlmod#modules

Bazel Central Registry

The Bazel team hosts a repository, https://github.com/bazelbuild/bazel-central-registry which is a database of published versions of Bazel Modules. Let’s find a Bazel-level dependency to add. Go to the web interface for the Bazel Central Registry: https://registry.bazel.build Search for whatever interests you. It should be something new.

Use the button to copy the text from the “Install” code block and paste it in MODULE.bazel. Now you can ask Bazel to fetch that package:

% bazel fetch @some_module//path:all

Or fetch everything, which will fetch just the added dependency if you’ve made no other changes:

% bazel fetch ...

Bazel creates a MODULE.bazel.lock file as well, which captures the specific versions of dependencies including transitive dependencies, making the module resolution reproducible. This file should be checked in.

Language dependencies

For most languages, Bzlmod delegates to other language-specific package managers.

Python: pip
JavaScript: pnpm
Java: Coursier / Maven / Gradle
Go: the go tool

C/C++ doesn’t really have a popular “package manager”, so the Bazel Central Registry is becoming one, in addition to a registry of Bazel modules.

In most cases, under Bazel we’ll still use the canonical files for declaring these dependencies, though under Bazel they should always be “pinned” for reproducible builds. Why? We want to preserve interoperability with existing tools as much as possible, such as editors and static analysis tools, and these understand the idiomatic files for the language. It’s possible to go “pure Bazel” and omit the language-specific files but Aspect doesn’t recommend it.

More about lockfiles“Pinned” means direct and transitive dependency versions are always exactly specifiedThis typically includes integrity hashes for supply chain security.A “good” lockfile supplies all the information Bazel rules need to reproduce the package manager’s logic for downloading dependencies and laying them out on disk.

The steps to do this vary a bit between languages, but they all have the following rough outline:

You may leave the developer’s constraints alone, but do use semantic versioning ranges. For example:
- our logger/frontend/package.json allows any version of http-server
- our requirements.txt allows any version of requests
Pin transitive dependencies to a constant version
- These are generally written to a separate “lock” file.
Mirror that dependency list into Starlark
- This allows Bazel to manage the dependencies itself.
Add code to expose external repositories for use by BUILD targets
- The instructions for each language should tell you how to do this.

Finding documentation for bzlmod

In practice, you’ll find that not all rules do a good job of documenting bzlmod usage yet. You can get a hint by finding the tests for a ruleset. On https://registry.bazel.build, click the “View registry source” link for a module, and open the presubmit.yml file. You’ll find a path to some sub-folder where a test lives. These are executable examples, so they give us a clue how the module is used. For example,

bcr_test_module:
    module_path: 'e2e/bzlmod'

Then you’d navigate to the /e2e/bzlmod folder in the ruleset repository, and there will be something that is guaranteed to work.

Exercise: add some language-specific dependencies

Bazel’s reproducibility can only be as good as the information it’s given. Each external package manager has a feature to pin the dependencies. The /README.bazel.md in the bazel-examples repo explains how to manage third-party package dependencies. You should be sure to have similar documentation for developers in your repository. The goal is to produce the following files, for the languages you care about:

go.mod -> go.sum
package.json -> pnpm-lock.yaml
requirements.txt -> requirements_lock.txt
Java sources -> maven_install.json
Package.swift -> Package.resolved

You’ll have to read the documentation for the ruleset you use to figure out an approach to do this, and also add to the MODULE.bazel file by searching the Registry and following ruleset install instructions. Refer to the 100-series courses for more details about language-specific developer workflows.

Configuring the downloader

Bazel’s downloader is full-featured, and you can use it to block undesired network access, fetch via your corporate proxy or artifact repository, and more. Beware though, that some rulesets use the Bazel downloader while others do not.

Eager fetches

We mentioned the eager semantics of load in Eager load. This is true for package managers and external packages too. Developers shouldn’t need to fetch things they don’t use. A developer working in one language shouldn’t be blocked waiting to download toolchains for some other language. Most of the time developers will not have a reason to fetch everything, but rather just let fetches happen automatically as needed.

In repository rules

These happen for every single build regardless of the dependency graph or which targets the user requests. Bazel must evaluate the complete WORKSPACE and MODULE.bazel files to understand what third-party dependencies exist for the build. Let’s say the WORKSPACE file contains this content:

load("@rules_python//python:pip.bzl", "pip_parse")

pip_parse(
   name = "my_deps",
   requirements_lock = "//path/to:requirements_lock.txt",
)

load("@my_deps//:requirements.bzl", "install_deps")
install_deps()

Because the highlighted line has a load statement, the my_deps repository is requested at loading time, and so the pip_parse implementation will run. If it uses a hermetic python interpreter, then that interpreter must be built or fetched for any build.

Exercise: Fetch

Let’s verify we have one language working, by using bazel fetch to get one of the dependencies.

​bzlmod: Bazel’s package manager

​Bazel Central Registry

​Language dependencies

​Finding documentation for bzlmod

​Exercise: add some language-specific dependencies

​Configuring the downloader

​Eager fetches

​In repository rules

​Exercise: Fetch

bzlmod: Bazel’s package manager

Bazel Central Registry

Language dependencies

Finding documentation for bzlmod

Exercise: add some language-specific dependencies

Configuring the downloader

Eager fetches

In repository rules

Exercise: Fetch