Establish Connectivity to Databricks

This article describes how to connect to Databricks in order to load data into Datasets and Reports in Metric Insights.

1. Create a New Data Source

Access Admin > Collection & Storage > Data Sources

  1. [+ New Data Source] 
  2. Select SQL as a type of new Data Source
  3. [Next Step]

2. Complete Data Source Creation

  1. Name: Give the Data Source a descriptive name
  2. Auth Type: Select the "Password" type of authentication
  3. To obtain Data Source Username and Data Source Password generate an Access token in the User Settings page of Databricks account
  4. Host name, Database name and Port fields will disappear after entering the JDBC string
  5. JDBC driver: Choose "Databricks" from the drop-down menu
  6. JDBC string: Insert the JDBC URL from the page of created SQL Warehouse in the Databricks account
    • "ConnCatalog"and "ConnSchema" parameters can be fount in the data Explorer of the Databricks account
  7. [Save]

NOTE: Be aware, that Databricks driver doesn't have full compatibility with Java 21. To use Arrow serialization with the Databricks JDBC driver, disable Arrow by setting EnableArrow=0 in the connection string. Disabling Arrow will not affect the functionality of the JDBC driver, but it may affect performance and memory usage.

3. Check the Connection

[Test Connection] to check if the system connects to the database successfully.

What's Next?

Create Dataset from Databricks