Page Comparison

...

At the time of writing, our Bazel builds and tests in CI all occur in a bespoke Docker image, cockroachdb/bazel. This image is different from the normal builder image and, notably, has a lot less stuff installed on it. This is intentional, and makes it substantially easier to identify missing dependencies.If a test is tagged broken_in_bazel (see the BUILD.bazel for your package), it’s not included in the test_suites defined in pkg/BUILD.bazel, and therefore isn’t run in CI. A near-term goal is to fix all broken tests and remove that tag to increase our test coverage.

Reproducing a broken test

A test failure can be reproduced locally with bazel test; for example, if the package in question is pkg/path/to/dir, you can run bazel test pkg/path/to/dir:dir_test. (If you want to use dev, you can all or equivalently ./dev test pkg/path/to/dir; be aware that dev is in a pre-alpha state. ) You may want to also set the --test_env=GO_TEST_WRAP_TESTV=1 command-line flag, which will cause the tests to be more verbose (like go test's -test.v flag). Consider running the test in the cockroachdb/bazel Docker image to avoid a scenario where you get a test working locally , but it fails in CI – check build/teamcity-bazel-support.sh for the latest image: to run the builder image, use ./dev builder.

Failed tests will be displayed prominently:

...

The test’s logs are in the file that the output mentions; if you find it more convenient, you can find the same logs in the _bazel-/testlogs directory in your workspace (in this case, that file would be at _bazel-/testlogs/pkg/internal/rsg/yacc/yacc_test/test.log).

...

• The helper library at pkg/build/bazel provides some general-purpose utilities:

  • BuiltWithBazel() returns true iff this test was built with Bazel. (The other functions in this library will panic unless this returns true, so always check that you’ve BuiltWithBazel first.) You can use this to perform some Bazel-only logic, for example:

    Code Block
    language go
    if bazel.BuiltWithBazel() { // Bazel-only logic } else { // `go test`-only logic }

  • Runfile() returns the absolute path to a file you’ve depended on:

    Code Block
    if bazel.BuiltWithBazel() { // NOTE: This will panic if you don't have `//pkg/sql/parser:sql.y` in your `data` dependencies. runfile, err := bazel.Runfile("pkg/sql/parser/sql.y") if err != nil { panic(err) } }

  • See pkg/build/bazel/bazel.go (and the library that this one wraps, at vendor/github.com/bazelbuild/rules_go/go/tools/bazel) for full documentation.

• If your test contains a testdata/ directory, you can use testutils.TestDataPath() to easily get a path to any file in testdata – for example, testutils.TestDataPath(t, “a.txt”) to locate the file testdata/a.txt. Note that Gazelle auto-generates a dependency on the testdata files for you in this case.

Using the Go toolchain in the sandbox

...

The make build (and the way that it’s triggered in CI) is more or less the “source of truth” for how Cockroach is built in different configurations, including tests. It’s possible that something’s been missed in how that build configuration has been “translated” to Bazel. If you suspect the build configuration needs to be changed in some way, contact a member of the Dev-Inf team.

...

geos

Some tests require the geos library. Further investigation is needed here: https://github.com/cockroachdb/cockroach/issues/61664 , https://github.com/cockroachdb/cockroach/issues/61665 You can declare a dependency on geos by adding “//c-deps:libgeos" to the data for your tests.

Other concerns

Test size

In Bazel, all tests can be annotated with a “size”. This implies a default timeout (you can also set a custom timeout). We have four test suites, one for each size – //pkg:small_tests, //pkg:medium_tests, //pkg:large_tests, //pkg:enormous_tests. The test suites each run all the tests in-tree for the given size. //pkg:all_tests contains all of the tests from the four test suites above.

Make sure your test is appropriately sized when you run it. Bazel will tell you how long it takes your test to run – as a rule of thumb, small tests should be < 30s (< 10s is better!), medium < 2m, large < 5m, and enormous anything longer.

...