Skip to main content
Databricks is a unified data intelligence platform.

Prerequisites

Setup

Environment Variables

Add the following to a .env file in your Cube project:

Service principal (OAuth M2M)

To connect as a Databricks service principal instead of with a personal access token, set the OAuth client ID and secret of the service principal. The service principal needs CAN USE on the SQL warehouse in the JDBC URL and access to the data Cube queries (with Unity Catalog, USE CATALOG, USE SCHEMA and SELECT). Requires Cube v1.3.20 or later.
When CUBEJS_DB_DATABRICKS_OAUTH_CLIENT_ID is set, Cube authenticates with OAuth and ignores CUBEJS_DB_DATABRICKS_TOKEN. With OAuth, Cube ignores AuthMech, UID and PWD in the JDBC URL and configures authentication itself, so set both environment variables and leave OAuth2ClientID, OAuth2Secret and Auth_Flow out of the URL.

Cube cloud platform

The Databricks data source form sets the client ID, client secret and JDBC URL for you. It is shown when you create a deployment and on the deployment’s Settings → Data Sources page:
  1. Select Service principal (OAuth M2M) authentication. Access token authentication is the default.
  2. Enter the OAuth client ID and OAuth client secret of the service principal.
  3. Enter the Databricks JDBC URL and, optionally, the Databricks Catalog.
A data source that already has an OAuth client ID opens with service principal authentication selected. Switching between the two methods and saving removes the other method’s credentials from the deployment.

Docker

Create a .env file as above, then extend the cubejs/cube:jdk Docker image tag to build a Cube image with the JDBC driver:
You can then build and run the image using the following commands:

Environment Variables

Pre-Aggregation Feature Support

count_distinct_approx

Measures of type count_distinct_approx can be used in pre-aggregations when using Databricks as a source database. To learn more about Databricks’s support for approximate aggregate functions, click here.

Pre-Aggregation Build Strategies

To learn more about pre-aggregation build strategies, head here.
By default, Databricks JDBC uses a simple strategy to build pre-aggregations.

Simple

No extra configuration is required to configure simple pre-aggregation builds for Databricks.

Export Bucket

Databricks supports using both AWS S3 and Azure Blob Storage for export bucket functionality.

AWS S3

To use AWS S3 as an export bucket, first complete the Databricks guide on connecting to cloud object storage using Unity Catalog.
Ensure the AWS credentials are correctly configured in IAM to allow reads and writes to the export bucket in S3.

Google Cloud Storage

When using an export bucket, remember to assign the Storage Object Admin role to your Google Cloud credentials (CUBEJS_DB_EXPORT_GCS_CREDENTIALS).
To use Google Cloud Storage as an export bucket, first complete the Databricks guide on connecting to cloud object storage using Unity Catalog.

Azure Blob Storage

To use Azure Blob Storage as an export bucket, follow the Databricks guide on connecting to Azure Data Lake Storage Gen2 and Blob Storage. Retrieve the storage account access key from your Azure account and use as follows:
Access key provides full access to the configuration and data, to use a fine-grained control over access to storage resources, follow the Databricks guide on authorize with Azure Active Directory. Create the service principal and replace the access key as follows:

SSL/TLS

Cube does not require any additional configuration to enable SSL/TLS for Databricks JDBC connections.

Databricks Metric Views

To publish deployed Cube views as native Databricks Metric Views, see the Databricks Metric Views guide. This integration is in preview.

Additional Configuration

Cube Cloud

To accurately show partition sizes in the Cube Cloud APM, an export bucket must be configured.