Page Index
- Raphael ‘kena’ Poss (Deactivated)
Owned by Raphael ‘kena’ Poss (Deactivated)
Feb 23, 2019
1 min read
Loading data...
Space Index
|
|||||||||||||||||||||||||||||||
0-9 |
AAdding, Updating and Deleting external dependenciesCockroachDB uses vendoring to capture all its dependencies at a specific version. If your project requires an external package that is not yet captured, you will need to add/update the dependencies. When ADDING or UPDATING dependencies, the update needs 4
|
||||||||||||||||||||||||||||||
BBackporting a change to a release branchAll development starts on the master branch. In most cases, to add a commit to a patch release (or a beta/rc after a release branch has been created), you should submit a PR to master first and get it reviewed and merged there. PRs should be eligible for
Ben's Go Emacs Setup
(This information was current as of 1/11/2016. As of 2022, it's rotted pretty badly, so I suggest looking elsewhere -Ben) In case you're interested, here's what I'm using in emacs now. I think these instructions will be at least somewhat repeatable. 1. Fi
Bors Merge Bot
Overview To guard against merge skews we merge pull requests using a bot. Craig is that bot, an instance of bors-ng https://github.com/bors-ng/bors-ng. It compiles approved PRs as if they were merged, and merges them only after the CI on the original PR p
Building and running tests
To build and run tests, just run make To run just presubmit checks (golint etc): make check Running the tests for a package: make test PKG=./sql Verbose test output: make test PKG=./sql TESTFLAGS='-v' Running tests, showing which tests are run and test fa
Building from source on macOS
This guide is for users intending to develop CockroachDB. To USE CockroachDB consult these docs instead: - cockroachlabs.com/docs/install-cockroachdb.html https://www.cockroachlabs.com/docs/install-cockroachdb.html - cockroachlabs.com/docs/start-a-local-c
|
CCan I still commit/merge code if there is still work left to do?Yes. Sometimes the work on a piece of code is sufficient for a proximate goal, although it does not yet reach the “ideal” target. We are OK with merging intermediate implementations, as long as the following holds: The implementation so far follows our en
CLA Assistant stuck on PR
If your PR is unable to transition from the pending state, try these. X seems not to be a GitHub user The user.email belonging to the author is checked against the list of GitHub accounts that signed the CLA. The GitHub account used to submit the pull req
Cluster virtualization and Multi-tenant CockroachDB
Introduction and strategy Cluster virtualization is a new way to structure the CockroachDB technology that achieves isolation between logical clusters. This is most useful when we share a common distributed storage across competing customers, i.e. multi-t
CockroachDB
Welcome to the CockroachDB wiki space! This space is meant for technical documentation and guides specifically for contributors to CockroachDB itself. See also: CockroachDB code repository: https://github.com/cockroachdb/cockroach https://github.com/cockr
CockroachDB Release Cycle
In 2024 we moved from a major release every 6 months to a major release every 3 months. See Quarterly Releases Project (Spring FY25) for more details. Planning happens twice a year to soliciting input from users and developers, and takes the release sched
Code commenting guidelines
Comments are there to help the author of the code as well as current and future co-workers. Motivation and strategy Comments help yourself as much as your co-workers to remember the important bits: code you wrote 6 months ago might as well have been writt
Code coverage
Coverage is computed in a nightly teamcity build https://teamcity.cockroachdb.com/project.html?projectId=Cockroach_Coverage. This coverage ignores any automatically generated files such as protocol buffers as well as binaries that aren't necessary for run
Code Labs
Various efforts have been made from time to time to create code labs: how-to documents that aim to help people understand the code by asking them to modify it themselves. The code labs live here. We don't have enough code labs, but we'd really like more!
Code reviews
Codelab: SQL↔KV hacking This Codelab touches on the sql layer Insert node and how it communicates its writes to the KV layer. It also touches on lower-level (storage-level) concepts such as transactions and intents. As an excuse to touch interesting code points, we’ll hack toget
Collecting a "debug zip" manually
When asked for help, the CockroachDB team often requests the output of the command cockroach debug zip. This utility automatically scrapes metadata about a cluster which can aid with debugging all kinds of issues. Sometimes, using cockroach debug zip is n
Community Code of Conduct
The CockroachDB community’s Code of Conduct can be found at the following location: https://github.com/cockroachdb/code-of-conduct https://github.com/cockroachdb/code-of-conduct
Contributing to CockroachDB
We made CockroachDB source-available to empower developers to fix and extend the product to better meet their needs. Nothing thrills us more than people so passionate about the product that they're willing to spend their own time to learn the codebase and
|
||||||||||||||||||||||||||||||
DDatabase BackgroundThis list is a compilation of readings which are valuable to a general understanding of the operation of Cockroach. This list is extensive (but not exhaustive), don't feel you need to read everything here, it's provided as a way to drill down into topics
Debugging with Delve
Note: In the past, Delve has garnered a reputation for randomly panicking and not living up to the expectations of a debugger. However, a couple of years have passed since then, and it is now possible to attach to and debug running cockroach clusters, mak
Developing with Bazel
We’ve migrated the cockroach build system to Bazel https://bazel.build/. Bazel is a modern build system with better performance characteristics and correctness guarantees than we currently have with make/go build. Today, you can perform almost all day-to-
|
EEngineering standardsError concepts and handling in CockroachDB Error concepts Programmers do not usually like thinking about errors. When learning how to program, initially, programming assignments are silent about error handling, or at best dismissive. For many applications, the “best practice” for error handling is
Error handling basics
Error Types We prefer the use of the CockroachDB errors library at github.com/cockroachdb/errors https://github.com/cockroachdb/errors. This is a superset of Go's own errors https://golang.org/pkg/errors and pkg/errors https://github.com/pkg/errors. error
Evaluate the complexity of your project
A good way to find a project properly sized for a first time contributor is to search for open issues with the label "help wanted" https://github.com/cockroachdb/cockroach/labels/help%20wanted. We separate three levels of complexity for projects: Low comp
Execution tracing a SQL query
NB: “trace” or “tracing” herein refers to the go runtime’s tracing. This is not the distributed tracing, which is the instrumentation we add at the application level, although these certainly overlap in places. traceon / traceoff Adding traceon somefile.t
|
||||||||||||||||||||||||||||||
F |
GGetting and building CockroachDB from sourceNote that at least 4GB of RAM is required to build from source and run tests. Install prerequisites If you are on macOS, you may find the Building from source on macOS documentation useful. If using Bazel to build Cockroach (for master and v22.1 onward),
Git Commit Messages
When you're ready to commit, be sure to write a Good Commit Message™. What's in a Good Commit Message™? We follow most general engineering best practices from other projects using Git. Some additional guidelines specific to CockroachDB apply. Basic guidel
Git diff side by side
How to get a side by side diff with git diff or git difftool in the terminal. Alternative 1 Use icdiff, as instructed on blog.scottnonnenberg.com/better-git-configuration https://blog.scottnonnenberg.com/better-git-configuration/ install icdiff (from jeff
GitHub Issue Labels
CockroachDB's GitHub repo https://github.com/cockroachdb/cockroach uses a system of GitHub labels https://github.com/cockroachdb/cockroach/labels to organize issues. The labels are organized into groups based on a capital letter prefix. The labels are gro
Go (Golang) coding guidelines
This coding guide is inspired from the Uber Go style guide at https://github.com/uber-go/guide/blob/master/style.md https://github.com/uber-go/guide/blob/master/style.md, with modifications specific to Cockroach Labs. Introduction These guidelines are the
Go function argument patterns
Naive style: “require-everything” // Bad: brittle, hard to understand. MyFn("required arg", "foo", 0, nil, "bar", DefaultValue) // Still bad: more readable, but still brittle. If an argument is added // in the definition, or the order of arguments is chan
Guidelines for Cluster Settings, CLI flags and Env vars
Note: these are a work in progress; check with #technical-leads-council for questions/clarification. Cluster settings Appropriateness Guideline: Consider appropriateness of a cluster setting versus some other configuration mechanism. A behavior specific t
|
||||||||||||||||||||||||||||||
HHigh-complexity projectsThese projects involve fundamental changes to the way CockroachDB works. What’s important to remember here is that supporting external development of these projects will take a non-trivial amount of time from the Cockroach team that would otherwise be spe
How to ensure your code builds with Bazel
NOTE: This documentation is meant for people who are struggling with Bazel-only build failures. If instead you want a high-level guide to performing day-to-day development tasks using Bazel, check out the "Developing with Bazel" documentation. What is Baz
How to ensure your tests can run in the Bazel sandbox
See also: How to ensure your code builds with Bazel This documentation is a work in progress; feel free to add any useful stuff that you may have uncovered. Background When Bazel executes tests via bazel test, its behavior differs from a standard go test
How to investigate a Go test failure
note These instructions, while possibly still useful, are out of date as command line flags have changed and tools been phased out. Some of the incantations below have been updated, others have not. Please use common sense and update this document with im
|
IIssue and Epic Refs in Commits and PRsAmidst a large and ever-growing volume of changes, connecting work to product roadmap epics via GitHub Issue and Jira Epic references (Issue and Epic Refs) improves the ability to make the connection between product changes and roadmap epics for the docs
|
||||||||||||||||||||||||||||||
JJargon: CockroachDB words for newcomerssaasSEO: glossary, terms, vocabulary, dictionary This page is https://go.crdb.dev/jargon https://go.crdb.dev/jargon Focusing here on those words and acronyms you may encounter in meetings or documents or Slack threads, but won't necessarily find explained
|
K |
||||||||||||||||||||||||||||||
LLarge scale testingworkload Workload refers both to the package and the tool. The package is built around an abstraction, called Generator, and the tool has various bits for working with generators. Generator is abstraction for a related collection of tables with (optionall
Log and error redactability
Our customers routinely send crash reports and log files to us, but they want confidence that this data does not contain confidential information. How do we achieve this? For this purpose, the CockroachDB source code uses redactability. This is a crdb-spe
Low-complexity projects
These cover things like bug fixes and small enhancements that do not involve any major architectural or design decisions. You should feel free to submit these for review as you come across them. Find or create an issue. Write the fix, include the appropri
|
MMaintaining CockroachDBThis page collects resources useful to the CockroachDB maintainers.
Marking features as "not yet implemented"
Unimplemented errors occur when CRDB recognizes a SQL feature but it is not yet supported. These unimplemented errors can be used by the product team to help prioritize the roadmap. These errors are also useful to customers when they provide links to Gith
Medium-complexity projects
For these projects, you should: Proactively reach out to the Cockroach team and discuss your proposal. One of our engineers will provide feedback on your design. Occasionally, you will be asked to contribute an RFC, too, in which case you should follow th
Multiple GOPATHs
Note: with the switch to Bazel, the build system does not have the same dependency on GOPATH and you can check out and build as many clones of the cockroach repo as you want in different locations. This information is kept here as written for reference or
|
||||||||||||||||||||||||||||||
NNightly testsOverview Every night, a set of tests run as part of the TeamCity project Nightlies https://teamcity.cockroachdb.com/project.html?projectId=Cockroach_Nightlies&tab=projectOverview. These tests have a few common characteristics: They set up a temporary Cock
|
OOffline builds with BazelUnlike the make-driven build process which uses the checked-in sources in the vendor submodule, the Bazel build will download sources as needed. Under normal circumstances this should be transparent to the user, but if you are in an environment with no ne
Organization of code into packages
We use Go packages to delimit units of functionality. We generally like smaller packages with a tight scope, but not so small that components that are always updated together are pulled away from each other by package boundaries. When to create new packag
Organizing PRs and Commits
Short, simple projects produce a PR containing a single, small commit. However, we strongly encourage contributors to split up larger projects into multiple PRs or commits in order to make code reviews more effective. This happens mainly in two situations
Overview of all test frameworks
Any project usually needs a combination of multiple types of tests simultaneously: both unit and integration tests – the latter especially if there are user-facing features. Unit testing Unit tests are valuable to ensure that a single component or functio
|
||||||||||||||||||||||||||||||
PPage IndexProductivity Tips SourceGraph SourceGraph is a web-based source browser. It can enhance github with on-mouse-over information about types/function, links to jump to definition, language-aware searching. The enhanced repository can be accessed through sourcegraph.com/github
Profiling CockroachDB
go tool pprof is a profiling tool built into Go. It is useful for understanding hotspots in your code. One can profile a running cockroach binary, or a specific go benchmark. Basics This blog post https://blog.golang.org/profiling-go-programs will get you
|
Q |
||||||||||||||||||||||||||||||
RRadu's vim setupQuick and dirty tips for how to get started with vim and go. 1. Install pathogen https://github.com/tpope/vim-pathogen mkdir -p ~/.vim/autoload ~/.vim/bundle curl -LSso ~/.vim/autoload/pathogen.vim https://tpo.pe/pathogen.vim And add to the top of ~/.vimr
Rail's Neovim setup
I use AstroNvim https://astronvim.com/ as my neovim distro. The up to date config can be found in my dotfiles repo https://github.com/rail/stow-dotfiles/tree/main/.config/nvim. The config uses AstroNvim abstraction, but it can be translated into regular n
Rando scripts
The following code snippets are for general reuse. Some more scripts can be found here too: cockroachdb/scripts https://github.com/cockroachdb/scripts Configure git to fetch PRs (particularly nice with git log --decorate [try git config log.decorate auto]
Release notes
We publish detailed release notes https://www.cockroachlabs.com/docs/releases/ describing changes that impact external users of CockroachDB. Engineers provide "Release note" texts in their cockroach repo commits. These are then aggregated and edited by th
Reporting bugs in CockroachDB
I found a bug in CockroachDB! What should I do? If you believe your finding is security-related: please head to this page https://www.cockroachlabs.com/security/responsible-disclosure-policy/ and follow instructions. If you can reproduce the problem relia
Roachtest overview
Roachtest is a test framework for integration testing of CockroachDB. In a nutshell, it consists of: a test suite in the crdb repo (pkg/cmd/roachtest/tests) a framework that invokes roachprod under the hood (roachprod is our general-purpose manual testing
Roachtest vs TestServer/TestCluster
For testing CockroachDB we commonly use two separate frameworks: for unit tests (one bit of functionality at a time), we use the TestServer/TestCluster framework. for integration tests (using the entire CockroachDB server product as a customer would), we
|
SSQL Syntax↔semantics MenagerieThis page outlines the relationship between: SQL syntax clauses, from the PostgreSQL dialect Algebraic-relational operators, provided by theory Nodes in the data structure (memo) used to represent logical plans during optimizations, that we call "relation
Submitting your contribution
All contributors need to sign the Contributor License Agreement https://cla.crdb.dev/cockroachdb/cockroach. (Re-)Familiarize yourself with What is a Good CockroachDB PR. Create a local feature branch to do work on, ideally on one thing at a time. If you a
|
||||||||||||||||||||||||||||||
TTeamCity Continuous Integration (CI)Overview We use JetBrains TeamCity for various continuous integration and testing tasks. Our instance is available at teamcity.cockroachdb.com https://teamcity.cockroachdb.com - you may sign in with your GitHub OAuth credentials. TeamCity CI is deprecated
Testing CockroachDB
Overview of all test frameworks The following sub-page detail our test mechanisms further.
TestServer and TestCluster
Overview TestServer and TestCluster are two frameworks we built to create Go unit tests for CockroachDB, for tests that need a functional SQL layer and/or KV layer. TestServer simulates a single CockroachDB node. TestCluster simulates a multi-node Cockroa
Tracing operations with Jaeger, Zipkin and OpenTelemetry
CockroachDB has extensive verbose logging and distributed tracing instrumentation built-in. One way in which this instrumentation is useful is through 3rd party trace collectors like Jaeger and Zipkin. CRDB can be instructed to trace everything it does an
Troubleshooting CockroachDB
Here you can find resources about analyzing and understanding an existing CockroachDB cluster.
|
UUnderstanding CockroachDBHere you can find resources about learning and understanding CockroachDB and its ecosystem. Be sure to also check the online docs: CockroachDB product docs: https://www.cockroachlabs.com/docs/stable/ https://www.cockroachlabs.com/docs/stable/ Managed Cock
Updating PRs during review
During a review, the reviewer requests additional change to a PR. How to modify the PR to address reviewer comments? There are two main ways to do this: git commit --amend and git commit --fixup. Both are detailed below. The approaches proposed below refl
Using ccache to speed up builds
Note: With the switch to Bazel, ccache is much less valuable. You should not need to have it installed and installing it is likely to impact your build in other ways. Finally, we have fewer C/C++ dependencies than we used to (e.g. no more RocksDB), and th
Using viewcore to analyze core dumps
Viewcore https://github.com/jordanlewis/debug/tree/master/cmd/viewcoreis a tool for analyzing and exploring the memory in Go core dumps that was originally created by the Go team. It’s useful for debugging memory leaks, reproducible OOM issues, and worklo
|
||||||||||||||||||||||||||||||
V |
WWhat is a Good CockroachDB PRYou have made some changes to CockroachDB and you want to submit it to the CockroachDB team for inclusion in the main release. How does this work? In summary, we will look at the following: Is this your first time submitting a PR? Check Your first Cockroa
What to expect when you're expecting (someone to review your code at CRL)
At Cockroach Labs we take code reviewing seriously. This is for a number of reasons, some of which include: We have an open sourced repository and the quality of our code is a public reflection on our company. Databases are long-lived pieces of technology
Working as a reviewer
We're all humans First and foremost, it's important to remember that the code you're reviewing was written by another human. To that end, you should consider the human side of reviewing code before getting to the technical aspects (a good primer of how t
Working with Protobufs
CRDB uses gRPC to send messages across processes, which serializes structured data with protocol buffers. If you’re unfamiliar with gRPC https://grpc.io/docs/what-is-grpc/introduction/ and protobufs https://protobuf.dev/overview/, click those links to lea
|
||||||||||||||||||||||||||||||
X |
YYour first CockroachDB PRThis document is a long-winded guide to preparing and submitting your first contribution to CockroachDB. It's primarily intended as required reading for new Cockroach Labs engineers, but may prove useful to external contributors too. The development cycle
|
||||||||||||||||||||||||||||||
Z |
!@#$ |
||||||||||||||||||||||||||||||
Copyright (C) Cockroach Labs.
Attention: This documentation is provided on an "as is" basis, without warranties or conditions of any kind, either express or implied, including, without limitation, any warranties or conditions of title, non-infringement, merchantability, or fitness for a particular purpose.