Deployment and Operations

4 mins read

Installing and Configuring CockroachDB for Storing Data

Learn about CockroachDB datastore required installation and configuration for a deployment of the Cloudentity platform.

CockroachDB Datastore Overview

CockroachDB is a distributed SQL database that aims to make it simple to build, deploy, and operate globally-scaled applications. It is designed to be highly available, survive data center outages, and maintain consistent performance across a geographically distributed database cluster.

One of the key features of CockroachDB is its ability to automatically shard data across multiple nodes in a cluster, allowing it to scale horizontally as more nodes are added. It also supports automatic replication and failover, which helps to ensure that data is always available and protected against data loss. Overall CockroachDB is designed for building and operating applications that needs to be distributed at scale, and does not have single point of failure, providing a high-availability and fault-tolerance for the data storage layer.

Why CockroachDB

CockroachDB is the backbone of the Cloudentity platform’s data storage layer, powering the storage and management of all crucial data related to the platform’s operations. Cloudentity’s decision to use CockroachDB as its primary datastore is driven by the database’s ability to handle globally-distributed data with high availability, reliability, and consistency.

Cloudentity uses CockroachDB to store various types of data, such as users added to Cloudentity Identity Pools, configuration of client applications connected to a workspace, information about consent grants, and many more.

What Is Not Stored in CockroachDB

One exception is that Cloudentity does not use CockroachDB to store session data, access tokens and identity tokens (stored in Redis), and audit/analytics/metrics data (stored in TimescaleDB).

CockroachDB Installation

Remember

For the Cloudentity platform to work CockroachDB installation and configuration is required.

At Cloudentity, to install and configure CockroachDB, we use Helm - a popular package manager for Kubernetes that allows users to easily install and configure complex software such as CockroachDB on a Kubernetes cluster. By using Helm to install CockroachDB, users can take advantage of several benefits that make the process of deploying and managing CockroachDB much simpler and more efficient.

Firstly, Helm provides a convenient way to define and manage the configuration of CockroachDB, including the number of nodes, storage settings, and networking settings, in a single, easy-to-read file called a chart. This makes it easy to understand and modify the configuration of CockroachDB as needed.

Additionally, Helm provides the ability to manage and upgrade the CockroachDB deployment in a controlled and repeatable way, this means that any updates or upgrades to the CockroachDB software can be easily rolled out to the cluster in a predictable manner, avoiding any possible disruption to the service.

When you install the Cloudentity plafrom on Kubernetes using Helm Charts, you can see that the CockroachDB dependency is included in our kube-acp-stack Helm Chart.

Remember

Cloudentity does not guarantee full compability and support for using the below solutions.

If you do not want to host your own instance of CockroachDB, you can consider using one of following solutions based on the PostgreSQL:

or follow official CockroachDB documentation for:

CockroachDB Version Recomendation

Below recommendation is based on what we are running in the Cloudentity SaaS offering. For CockroachDB version supported in a given release, check its release notes.

Database: 23.2.2 Helm chart: 10.0.9

Supported versions of CockroachDB

  • 20.2.x
  • 21.x.x
  • 22.x.x

Configure CockroachDB Dependency

If you wish to configure the connection between the Cloudentity platform and CockroachDB, and configure the datastore installation itself, configure the values.yaml file for your Cloudentity deployment and apply the changes.

Configure Connection Between Cloudentity Platform and CockroachDB

If you need to configure the connection between the Cloudentity platform and CockroachDB:

  1. Refer to the sql section of the Cloudentity Platform Configuration Reference to learn about available configuration options.

  2. Change the configuration for the connection in the SQL client section of the Cloudentity Platform values.yaml file for your deployment.

  3. Apply the changes to your deployment.

CockroachDB Configuration Example

Cloudentity delivers a production-grade example derived from our own operational experiences. This example is readily accessible within the acp-on-k8s repository. Notably, this setup introduces the CockroachDB operator and subsequently deploys the CockroachDB database utilizing a custom resource.

We recommend utilizing this example as a foundational reference for your CockroachDB deployment. Get started with our quickstart guide.

Troubleshooting Issues with CockroachDB

If CockroachDB is not installed, you are able to see the following error in the logs for your Cloudentity platform deployment:

{"error":"failed to create database client: failed to connect to `host=acp-cockroachdb-public user=root database=defaultdb`: hostname resolving error (lookup acp-cockroachdb-public on 1.0.0.0:1: server misbehaving)","level":"fatal","msg":"failed to connect to the database"}

If CockroachDB is missconfigured and the connection between the Cloudentity platform and CockroachDB datastore is working incorrectly, you are able to see the following error in the logs for your Cloudentity platform deployment:

{"error":"failed to create database client: failed to connect to `host=crdb user=root database=defaultdb`: hostname resolving error (lookup crdb on 1.0.0.0:1: server misbehaving)","level":"fatal","msg":"failed to connect to the database"}
Updated: Oct 27, 2023