Page Comparison

...

dev is a light wrapper around Bazel that is well-suited to specific CRDB development tasks. Properly, when we say dev, we’re referring to the Go binary whose source lives in cockroach/pkg/cmd/dev, side-by-side with the Cockroach source. At the top level of the repo there is a wrapper script also called dev whose only job is to build pkg/cmd/dev and then invoke it with the passed-in arguments. This means that you can always type ./dev to run the version of dev at the commit you’ve checked out, which is typically what you want to do.

Note that dev is meant to supplement bazel rather than replace it. dev makes certain tasks easier than bazel, but simple builds and tests are easy enough to run via bazel directly.

Doctor, doctor

Before trying to build anything, run the following:

...

Start by building cockroach-short as follows:

./devbazel build //pkg/cmd/cockroach-short

(Note: the first time you run ./dev, dev will need to build itself, so be patient ) ./dev build short also works as an alias. The first thing dev does is tell you what bazel command it’s running. Bazel will pretty-print build output to the terminal and assuming the build is successful, dev will symlink the binary to ./cockroach-short for you.

You can also build cockroach{ccl,oss}:

./dev build //pkg/cmd/cockroach //pkg/cmd/cockroach-oss

As above, ./dev build cockroach cockroach-oss works as alias.

You can also build the full cockroach binary which includes

bazel build pkg/cmd/cockroach --config=with_ui

./dev build (or equivalently, ./dev build cockroach) is a synonym for this.

dev build is a light wrapper for bazel build that supports aliases for common targets (for example, ./dev build crlfmt instead of the harder-to-remember bazel build @com_github_cockroachdb_crlfmt//:crlfmt). dev also copies binaries out of the Bazel output directories for you into your workspace; for example, bazel build pkg/cmd/cockroach-short puts the binary in _bazel/bin/pkg/cmd/cockroach-short/cockroach-short_/cockroach-short, but if you dev build short the binary will be staged at ./cockroach instead.

Run ./dev help build for more information about what you can do with dev build. Note that you can pass additional arguments directly to bazel build by adding them after --:

# build "verbosely", outputting all commands as they are run
./dev build short -- -s

You can dev lets you cross-compile with the --cross option, as in:

...

--cross takes an optional argument which is the platform to cross-compile to: --cross=linux, --cross=windows, --cross=macos, --cross=linuxarm, --cross=macosarm. dev will symlink copy the built binaries into the artifacts directory in this case. Note that cross-building requires Docker. Cross-compiling should work on M1 Macs, but this support is experimental, so report issues if you should observe any.

For more debugging tips on building with Bazel, see “How to ensure your code builds with Bazel”.

Running tests

Run ./dev test with the name of one or more packages to execute all tests from those packagesYou can run all tests in a given package (for example, pkg/sql) with:

./devbazel test pkg/sql:all

bazel test pkg/sql/... will instead run all tests under the pkg/sql directory recursively, including all the tests under pkg/sql/types and pkg/sql/rowinfra among others.

If the test passes, you’ll see a brief message indicating which tests were run and their status:

...

You can examine that file to see the complete logs for the test.

./dev test is provided as a shorthand for bazel test with some additional conveniences. Run ./dev test with the name of one or more packages to execute all tests from those packages:

./dev test pkg/sql

Tips and tricks

• dev test has a --stress flag for running tests under stress and --race for running tests with the race detector enabled.

• Next to the test.log file produced by your test, you can find a test.xml file. This file contains specific information on each test run and its status, as well as timing information.

• The -v argument to dev test will result in more verbose logging as well as more detailed information written to the test.xml. You can make this the default behavior on your machine by adding test --test_env=GO_TEST_WRAP_TESTV=1 to your .bazelrc.user file.

• As with dev build, dev test allows you to pass additional arguments directly to Bazel by putting them after --: for example, dev test pkg/sql/types -- --verbose_failures --sandbox_debug.

• To get test results printed as tests are being run add -v -- --test_output streamed to the test command. Note that this reduces test parallelism.

• For more tips on debugging test failures, see “How to ensure your tests can run in the Bazel sandbox”

Other tasks

# Run benchmarksacceptance for pkg/sql/parser
tests
./dev bench pkg/sql/parser
# Open a container running the "bazelbuilder" image. Requires Docker.acceptance
# Run compose tests
./dev compose
# Run benchmarks for pkg/sql/parser
./dev builder bench pkg/sql/parser
# Generate code (run this before submitting your PR).
./dev generate
# Run lints
(WARNING: not all lints work in Bazel yet)
./dev lint
# logic tests!
./dev testlogic --files=$FILES --subtests=$SUBTESTS --config=$CONFIG
# Open a container running the "bazelbuilder" image. Requires Docker.
./dev builder

dev vs. make

This is a (non-exhaustive) 1-to-1 mapping of dev commands to their make equivalents. Feel free to add to this

General dev tips

Since ./dev always builds dev unconditionally before doing anything else, that means there will be a slight delay before your build or test actually begins. Due to caching this pause should never be too long, but if it’s annoying you or you’ve found a way to thrash the cache, you can do:

./dev build dev

...

The top-level dev script uses Bazel to build pkg/cmd/dev before running unless another dev binary with the same increasing integer ID has already been built. Generally dev will invoke the dev binary “as of” that commit, which should usually be the correct behavior. However, if the default behavior does not work for some reason, you can find all the built versions of dev under bin/dev-versions.

A (hopefully) fast and error proof dev workflow

...

• Bazel has a configuration file called .bazelrc. You can put a global configuration file at ~/.bazelrc or a per-repository file at .bazelrc.user in the root of your cockroach repo.

• Tired of running ./dev gen bazel? Set the ALWAYS_RUN_GAZELLE env-var to automatically run ./dev gen bazel before every dev test or dev build incantation. Note this does add a tiny delay – noticeable when iterating on small tests through dev test.

  • i.e. echo 'export ALWAYS_RUN_GAZELLE=1' >> ~/.zshrc

• If you have ccache installed, bazel will fail with an error like ccache: error: Failed to create temporary file for /home/alyshanjahani/.ccache/tmp/message_li.stdout: Read-only file system. To avoid this you should get the ccache links out of your PATH manually (i.e. uninstall ccache).

  • Alternatively, if you would like to use Bazel with ccache, you can enable support for writing outside the sandbox by adding the following to your $HOME/.bazelrc or <repo>/.bazelrc.user file:
    - For MacOS/Darwin:

    Code Block
    build --sandbox_writable_path=/Users/<USER>/Library/Caches/ccache/

    - For Linux:

    Code Block
    build --sandbox_writable_path=/home/<USER>/.ccache

If you’re using a different ccache directory (ccache --get-config cache_dir) point to that instead.

I just want to use Bazel directly 😠

...

dev vs. Bazel

You can always use bazel directly instead of going through dev but there are some things you might want to keep in mind:

• You should still ask dev doctor if your machine is up-to-snuff before you try to bazel build. The checks it performs aren’t dev-specific. dev doctor also sets up a local cache for you.

• dev prints out the (relevant) calls to bazel it makes before it does so. You can therefore run dev once just to learn how to ask Bazel to perform your build/test and then just directly call into bazel on subsequent iterations.

  • When running tests under stress, race, or --rewrite, dev does the legwork to invoke with the necessary flags with bazel. This involves running under another binary (stress), running with certain gotags (race), or allowing certain paths outside the bazel sandbox to be written to (testdata). Feel free to see the actual bazel command invoked and tweak as necessary.

• If you want to build with the UI, you must include the --config with_ui argument to bazel build. (dev takes care of this for you if you are using it.)

• In general, test targets are named after the package under test plus the suffix _test. For example: bazel test pkg/sql/parser:parser_test. Note that there are exceptions: pkg/roachpb:string_test is one, and more may be added later.

• If you want to build a test without running it, you must include the the --config test argument to bazel build. (dev takes care of this for you if you are using it.)When running tests under stress, race, --rewrite, dev does the legwork to invoke with the necessary flags with bazel. This involves running under another binary (stress), running with certain gotags (race), or allowing certain paths outside the bazel sandbox to be written to (testdata). Feel free to see the actual bazel command invoked and tweak as necessary.