For testing CockroachDB we commonly use two separate frameworks:

for unit tests (one bit of functionality at a time), we use the TestServer/TestCluster framework.
for integration tests (using the entire CockroachDB server product as a customer would), we use the roachtest framework.
- As a special case, integration tests that exercise the interactive (non-automatable) behavior of cockroachdb do not currently use roachtest.. Likewise, integration tests that exercise all types of client drivers do not use roachtest either. See the separate page on acceptance tests for details.

Similarities and Differences between the two frameworks

	TestServer/TestCluster	Roachtest

	TestServer/TestCluster	Roachtest
Can be used to start a 1-node cluster or multi-node cluster	yes to both
Can be configured to use a custom external configuration, i.e. a configuration that a customer may change in production. For example: TLS certificate directory, special cluster settings, etc.	yes to both
Can be configured to a custom internal configuration, i.e. a testing knob that injects a custom dependency to an internal component or a special behavior not available in release builds.	yes	no
Startup time until test cluster is ready for use by test	less than 300ms	more than 30s (100x slower) on first run, due to the need to build and/or download a cockroach binary after the first run, more than 3s (10x slower)
Suitable for unit tests	yes	limited: slow startup hinders fast development cycles exercises the entire `cockroach` binary so difficult to isolate individual components.
Suitable for integration tests for automatable product behavior and with standard SQL client drivers.	no - does not exercise external configuration code paths.	yes
Suitable for integration tests for non-automatable product behavior and/or with non-standard SQL client drivers.	no - see separate page on acceptance testing
Address Space Isolation	no	yes
Can use a release `cockroach` binary	no	yes
Can simulate mixed-version clusters and exercise mixed-version testing	limited (see discussion below)	yes

NOTE: TestServer/TestCluster shares the same address space (of the test binary) amongst server instances. This may result in surprising behavior when server settings are shared; e.g., testcluster: disallow sharing of Settings between nodes · Issue #112395 · cockroachdb/cockroach

When to use TestServer vs when to use Roachtest

One should opt to use TestServer/TestCluster when implementing unit tests that need a fully functioning SQL layer or KV layer.
- Unit tests are valuable to ensure that a single component or function operates as designed, regardless of how it was integrated from other components.
- Tests that do not need a full SQL layer and KV layer can exercise their components in a more focused manner without instantiating a full server via TestServer.
One would otherwise use Roachtest when implementing integration tests:
- Integration tests are valuable to exercise a product journey as experienced by an end-user.
- Integration tests are also the only way to reliably exercise external configuration mechanisms.
Currently, mixed-version testing is only easy to perform when using roachtest. So even when one wants to implement a “unit test”, as soon as one needs to also include mixed-version testing it is easier to achieve this using roachtest.
- But not the only way. See discussion below.

For mixed-version testing we currently have two frameworks:

for integration tests, there is a native and extensive mixed-version test runner inside roachtest. Refer to the roachtest docs for details.
for unit tests, we only have some limited support for mixed-version testing in the SQL logic test framework. It is achieved by downloading different versions of CockroachDB under the hood. Because of this, these mixed-version SQL logic tests can only really be run in CI.
It is still an open question as to how to perform mixed-version unit tests generally while keeping the test speed reasonable and high velocity on development cycles.