Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

For testing CockroachDB we commonly use two separate frameworks:

  • for unit tests (one bit of functionality at a time), we use the TestServer/TestCluster framework.

  • for integration tests (using the entire CockroachDB server product as a customer would), we use the roachtest framework.

    • As a special case, integration tests that exercise the interactive (non-automatable) behavior of cockroachdb do not currently use roachtest.. Likewise, integration tests that exercise all types of client drivers do not use roachtest either. See the separate page on acceptance tests for details.

...

  • for integration tests, there is a native and extensive mixed-version test runner inside roachtest. Refer to the roachtest docs for details.

  • for unit tests, we only have some limited support for mixed-version testing in the SQL logic test framework. It is achieved by downloading different versions of CockroachDB under the hood. Because of this, these mixed-version SQL logic tests can only really be run in CI.

  • It is still an open question as to how to perform mixed-version unit tests generally while keeping the test speed reasonable and high velocity on development cycles.