Repository Structure: Bazel packages

Goal: understand the Bazel configuration files for the “first-party” code in the repository. By the end of this section, you’ll be able to run bazel query and similar commands to explore Bazel’s dependency graph for at least one language.

DefinitionA Bazel “package” is a folder containing a BUILD or BUILD.bazel file, along with subfolders that don’t have one of these files. It is a “collection of related files and a specification of how to use them to produce output artifacts”.

The BUILD file is written in a subset of Starlark. Starlark is a configuration language with performance guarantees, and the BUILD file subset is roughly the declarative constructs. Things like for loops are not legal syntax in BUILD files, but list comprehensions are.

`glob`

The glob function allows you to use wildcard patterns to choose source files. However, it does come with some performance penalties. Using list comprehensions together with glob is a powerful way to stamp out targets, for example:

TESTS=glob(["testcases/*.json"])
[
    cc_test(
        name = testfile.replace("testcases/", "").replace(".json", ""),
        srcs = ["lookup-datatest.cc"],
        deps = [
            "//speller/lookup",
            "//third_party/nlohmann-json:json",
            "@googletest//:gtest_main",
        ],
    )
    for testfile in TESTS
]

Writing `BUILD` files

There are three typical ways to create and maintain BUILD files.

Automatically

The computer should be expected to maintain 80% of the BUILD files, since most of their content may be inferred from the source files. Product developers consider it a regression to have to repeat import statements as deps for Bazel’s benefit. Gazelle is a popular tool with extensions available for several languages. Add an import statement, run the tool, and the relevant BUILD file is updated to reflect it.

The configure subcommand is specific to Aspect CLI, providing a straightforward and uniform way to invoke build file generation. Projects using core Bazel only can direct developer to run a suitable script instead.

Machine-editing

You can use buildozer to script around printing and modifying BUILD file content, which is an essential skill for doing repository-wide refactoring. Buildozer is purely syntactic, operating on the Starlark Abstract Syntax Tree (AST). This is convenient if you want to see what the user typed, before loading and macro expansion occur. It’s also guaranteed to be fast, while loading might take a long time.

There’s a dedicated aspect print command to make this feature easier to access.

By Hand

While the situation has improved tremendously in recent years, 20% of BUILD files are typically hand-edited, because:

They do things not described by the source files, or
They are for languages with no Gazelle support available (yet)

Eager `load`

Warning: the load statement is evaluated immediately in the loading phase, and extends to every reachable transitive starlark file, including those defined in external repositories. This can lead to long, unnecessary downloads! It’s also difficult for product engineers to understand how the graph shape contributes to slow builds. When a package requires loading from many different languages or extensions, this can be a smell which indicates the sources should be re-organized. For example you might want a different subfolder for each language. In this example, a BUILD file loads from @npm:

load("@npm//@bazel/typescript:index.bzl", "ts_project")

package(default_visibility = ["//visibility:public"])

ts_project(
    name = "a",
    srcs = glob(["*.ts"]),
    declaration = True,
    tsconfig = "//:tsconfig.json",
    deps = [
        "@npm//@types/node",
        "@npm//tslib",
    ],
)

filegroup(name = "b")

Even if a developer only asks Bazel to build the filegroup named b, the load statement means that the @npm repository must be fetched. Here’s a fictitious worst-case example, defaults.bzl

# top-level loads from many languages / external repos
load("//tools/cc:defs.bzl", "cc_library")                        # local C++ helper macros
load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_test")           # external C++ rules (external repo -> may trigger repo fetch)
load("@rules_java//java:defs.bzl", "java_library", "java_test")  # external Java rules
load("@rules_python//python:defs.bzl", "py_library")             # external Python rules
load("@io_bazel_rules_go//go:def.bzl", "go_library")             # external Go rules
load("@npm//:nodejs.bzl", "nodejs_library")                      # npm / nodejs rules
load("@rules_rust//rust:defs.bzl", "rust_library")              # external Rust rules
load("//tools/sh:sh_rules.bzl", "sh_binary")                     # local shell helper rules
load("//tools/proto:proto_helpers.bzl", "generated_proto")       # local proto helper (may call codegen macros)
load("@com_google_protobuf//:protobuf.bzl", "proto_library")     # external proto support

# Legacy/third-party macro that may do expensive top-level work inside its .bzl
load("//third_party/legacy:legacy_macros.bzl", "legacy_wrap")    # BAD SMELL if legacy_macros has heavy top-level logic

# configuration helper that computes values at load-time (potentially expensive)
load("//tools/opt:dynamic_config.bzl", "resolve_dynamic_flags")  # calling this at load triggers eval of that .bzl

The correct approach here is to break up the files into subfolders that can each use a subset of the loads.

​glob

​Writing BUILD files

​Automatically

​Machine-editing

​By Hand

​Eager load

`glob`

Writing `BUILD` files

Automatically

Machine-editing

By Hand

Eager `load`