How to ensure your code builds with Bazel

NOTE: This documentation is meant for people who are struggling with Bazel-only build failures. If instead you want a high-level guide to performing day-to-day development tasks using Bazel, check out the "Developing with Bazel" documentation.

What is Bazel? Why do I have to think about it?

Bazel is a modern build system with better performance and correctness guarantees than our old build system, make. Since we’re currently only partly through the migration to Bazel, we’re currently in a limbo where we need to ensure CockroachDB can build with both Bazel and make until we’re ready to switch over entirely.

If you’re interested in watching or contributing to the progress of the Bazel migration, you can check out the project tracker on GitHub.

CI is failing because the Bazel build is broken. What do I do?

Compared to make, Bazel is much more rigorous about properly declaring dependencies between build targets. Most issues with the Bazel build can be solved by identifying and fixing the missing dependencies.

dev generate bazel

We use an open-source tool called Gazelle which automatically fixes up all the dependencies in the workspace for all our Go and protobuf code. You can run Gazelle with ./dev generate bazel. (bors will not let you merge unless this command can be run cleanly, so you may want to make running this a part of your normal workflow.) If Gazelle notices changes, this will result in updates to BUILD.bazel files in tree, or to the root-level WORKSPACE or DEPS.bzl files. Then, re-run the Bazel build to ensure this fixes the failure. Note that ./dev generate bazel --mirror is required when vendoring new dependencies.

Updating dependencies for protos

There is a class of Bazel build errors that Gazelle can’t handle, and manual intervention is necessary. In this case you typically need to manually add a dependency to the corresponding BUILD.bazel file for the package that’s failing to build. Here’s a current example (at time of writing).

In pkg/sql/execinfrapb/processors_bulk_io.proto you can see the following line of code:

repeated string partition_addresses = 1 [(gogoproto.customtype) = "github.com/cockroachdb/cockroach/pkg/ccl/streamingccl.PartitionAddress",(gogoproto.nullable) = false]

The protobuf compiler will turn this into a .pb.go file that imports the package github.com/cockroachdb/cockroach/pkg/ccl/streamingccl; but Gazelle doesn’t know that, so ./dev generate bazel won’t insert the necessary dependency. This manifests as an error when you try to bazel build:

compilepkg: missing strict dependencies:
	/private/var/tmp/_bazel_ricky/be70b24e7357091e16c49d70921b7985/sandbox/darwin-sandbox/2323/execroot/cockroach/bazel-out/darwin-fastbuild/bin/pkg/sql/execinfrapb/execinfrapb_go_proto_/github.com/cockroachdb/cockroach/pkg/sql/execinfrapb/processors_bulk_io.pb.go: import of "github.com/cockroachdb/cockroach/pkg/ccl/streamingccl"

We fix this by adding the dependency ourselves, in pkg/sql/execinfrapb/BUILD.bazel:

go_proto_library(
    name = "execinfrapb_go_proto",
    ...
    deps = [
        "//pkg/ccl/streamingccl",  # keep

The # keep comment is very important as it tells Gazelle not to remove a dependency it would otherwise think is unnecessary. Generally, for missing dependencies on Go code in the github.com/cockroachdb/cockroach repo, just strip out the prefix and replace it with // to get Bazel to look in the right place (for example, here we’ve replaced github.com/cockroachdb/cockroach/pkg/ccl/streamingccl => "//pkg/ccl/streamingccl").

After you manually update your dependencies and ensure the build has succeeded, it’s a good idea to re-run ./dev generate bazel to have Gazelle fix everything up. This will let you know if you forgot any # keep comments.

CI is failing because a Go test is broken in Bazel. What do I do?

See full article here!

First, make sure you follow all of the above advice. If building the test binaries succeeds but running the test fails, and that same test failure doesn’t show up in the non-Bazel CI jobs, you’ve probably identified a case where Bazel’s test sandboxing has caused a breakage. There’s no one-size-fits-all solution to this; fixes will vary wildly depending on what the problem is and how it manifests. Generally, we recommend you ask for help (see below).

For integration tests, lint tests, or other heavyweight tests, it might be impractical to land Bazel support in the same PR, in which case it can be appropriate to tag your test as broken_in_bazel. Make sure to follow up with a member of the developer infrastructure team when you do so we can make sure the migration to Bazel gets triaged appropriately.

How to ask for help

If you don’t know how to solve a build or test failure, engage with a member of the developer infrastructure team. If you’re a Cockroach Labs employee, you can reach out to #bazel on Slack. External contributors can ping #contributors on the CockroachDB Community Slack, or you can engage the reviewer on your pull request for help.

Miscellaneous tips

See the comment at the top of DEPS.bzl for help on injecting temporary changes to dependencies. This is useful if you want to test changes to dependencies before upstreaming them or if you want to inject debugging logic into third-party code.
Generated files handled by go:generate are not natively generated in the Bazel sandbox, so introductions of new calls to go:generate can break the Bazel build or otherwise lead to unpredictable or incorrect behavior. If you add a new go:generate somewhere, make sure the equivalent logic to generate the file is present in the Bazel build as well. Follow the directions above to ask for help if you don’t know how to do this.
As mentioned here, if building on MacOS Bazel requires the full version of XCode: Building from source on macOS