Overview

TestServer and TestCluster are two frameworks we built to create Go unit tests for CockroachDB:, for tests that need a functional SQL layer and/or KV layer.

• TestServer simulates a single CockroachDB node.

• TestCluster simulates a multi-node CockroachDB cluster. Each node is simulated using one TestServer.

By default, TestServer uses in-RAM storage (not persisted) to make tests faster.

For integration tests, consider using roachtest instead. See: the pages Overview of all test frameworks and Roachtest vs TestServer for details.

For other unit tests that do not need a full SQL or KV layer, consider a more narrow unit test that only instantiates the components it cares about. This will make the test run faster.

Here are the main differences between a regular CockroachDB node (e.g. one started via cockroach start) and one simulated via TestServer:

...

func StartServerOnly(t TestFataler, params base.TestServerArgs) TestServerInterface

type TestServerInterface interface {
  TestServerController
   // ApplicationLayer returns the interface to the application layer that is
  // exercised by the test.
  ApplicationLayer() ApplicationLayerInterface

  // SystemLayer returns the interface to the application layer
  // of the system tenant.
  SystemLayer() ApplicationLayerInterface

  // StorageLayer returns the interface to the storage layer.
  StorageLayer() StorageLayerInterface

  ...
}

// TestServerController defines the control interface for a test server.
type TestServerController interface {
   ...
  // Stopper returns the stopper used by the server.
  Stopper() *stop.Stopper
  ...
}

The result of StartServerOnly is TestServerInterface, which contains the following two main methods: ApplicationLayer() and StorageLayer(). They refer to the following architectural diagram:

In short, ApplicationLayer() gives the test code a handle to the application layer of CockroachDB, containing the SQL and HTTP components; while StorageLayer() gives the test a handle to the storage layer with KV (replication, transactions) and lower level storage (Pebble).

In these two interfaces we see the following methods, for example:

// ApplicationLayerInterface defines accessors to the application
// layer of a test server.
type ApplicationLayerInterface interface {
  ...
  // SQLConn returns a handle to the server's SQL interface, opened
  // with the 'root' user.
  // The connection is closed automatically when the server is stopped.
  SQLConn(t TestFataler, dbName 𝓢) *gosql.DB
  ...
}

// StorageLayerInterface defines accessors to the storage layer of a
// test server. See ApplicationLayerInterface for the relevant
// application-level APIs.
type StorageLayerInterface interface {
  ...
  // LookupRange looks up the range descriptor which contains key.
  LookupRange(key roachpb.Key) (roachpb.RangeDescriptor, ⊙)
  ...
}  

Finally, a few methods are implemented by TestServerInterface directly by inheriting from TestServerController; for example Stopper() is part of TestServerController. They correspond to “orchestration-level” control of the server.

Automatic cluster virtualization

Background - quick intro to cluster virtualization

CockroachDB has supported cluster virtualization since v20.x, as the underlying technology to CC Serverless. Cluster virtualization corresponds to the virtualization of the application layer (as per the diagram above). It’s possible for the SQL and HTTP components to be fully encapsulated into a “virtual cluster” such that multiple virtual clusters can exist side-by-side on top of the same storage layer.

This corresponds to the following diagram (theoretical):

In this diagram, the “System” box is a special SQL interface to the storage layer which exists outside of virtualization; it corresponds to the “control layer” of the virtualization system and can be used to set parameter across all virtual clusters.

TestServer and automatic cluster virtualization

By default, TestServer automatically randomizes the server architecture between one of the following choices:

That is, a virtual cluster is started probabilistically inside the TestServer and the TestServer’s .ApplicationLayer() accessor is configured to point to it.

Tests should be implemented to primarily access SQL/HTTP via the ApplicationLayerInterface returned by .ApplicationLayer() to automatically get coverage with and without cluster virtualization.

Meanwhile, regardless of the randomization, the following invariants are always true:

• .StorageLayer() always points to the storage layer inside TestServer.

• .SystemLayer() always points to the special system interface inside TestServer (previously known as “system tenant” in previous versions of CockroachDB).