# Connecting to Databricks

## Connect to Databricks

Connect your Databricks account to OneLens to import usage and billing data for cost analysis, anomaly detection, and optimization insights.

{% hint style="info" %}

#### Before you begin

Ensure the following:

* You have **Account Admin** access in Databricks
* Your workspace is **Unity Catalog-enabled**
* You can create Service Principals and SQL Warehouses
  {% endhint %}

{% hint style="info" %}

#### Required Permissions

The Service Principal used for this integration must have the following permissions:

#### SQL Warehouse

* `CAN USE` on the SQL Warehouse

#### Unity Catalog (System Tables)

Grant **Data Reader** access on:

* `system.billing`
* `system.compute`
* `system.access`
  {% endhint %}

***

## Configuring Databricks

Complete the following steps in your Databricks account before integrating with OneLens

{% stepper %}
{% step %}

### Collect account and workspace details

* Log in to the Databricks console
* Copy your **Databricks Account ID**

<figure><img src="/files/SExmWQ0bB4zEdiaseLZ1" alt=""><figcaption></figcaption></figure>

* Navigate to **Workspaces** and select your workspace
* Copy the **Workspace URL**

<figure><img src="/files/aPPMC73Hbwj5hyHOKNxh" alt=""><figcaption></figcaption></figure>

* Open the workspace
  {% endstep %}

{% step %}

### Create a Service Principal

* Go to: **Settings → Identity and access → Service principals**
* Click **Add service principal**

<figure><img src="/files/m2JnmXQRolZ32Dr7QDT3" alt=""><figcaption></figcaption></figure>

* Enter a name (for example, `onelens-integration-sp`) & click on add
* Open the Service Principal and go to the **Secrets** tab
* Click **Generate secret. When the secret expires, you’ll need to create a new one and reconfigure integration with OneLens.**

<figure><img src="/files/lYtBtsGBqXVMhga6LSya" alt=""><figcaption></figcaption></figure>

* Copy the following and save to use in OneLens :
  * Client ID
  * Client Secret

<figure><img src="/files/68Se0CeduXn8HbV52er0" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Create a SQL Warehouse

* Navigate to: **SQL → SQL Warehouses**
* Click **Create warehouse**

<figure><img src="/files/Yc1vvWAhcsomAlG0HtSo" alt=""><figcaption></figcaption></figure>

* Configure:
  * Type: Serverless
  * Size: **2X-Small**
* Create the warehouse
* Copy the **SQL Warehouse ID**

<figure><img src="/files/vy3qq10zCp4ZF4LfJoRh" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Grant warehouse access

* Open the SQL Warehouse
* Go to **Permissions**
* Add the Service Principal
* Grant `CAN USE` permission

<figure><img src="/files/SBqdGlKjxZAbAjzSwAhS" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Grant system table access

* Navigate to **Catalog → system**
* Grant **Data Reader** access to the Service Principal on:
  * `system.billing`
  * `system.compute`
  * `system.access`

<figure><img src="/files/J0v9b350PkHZzvdbsgOF" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

## Connecting Databricks in OneLens

Complete the following steps in OneLens.

{% stepper %}
{% step %}

### Open the integration flow

1. Navigate to **OneLens → Integrations → Databricks**
2. Click **Connect Databricks**

<figure><img src="/files/V8nH01fNx5IcMOXI9HNA" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Enter account details

1. **Integration Name**
2. **Databricks Account ID**
3. Workspace URL
4. **SQL Warehouse ID**

<figure><img src="/files/QwMRtUK25RE43oP6DLWG" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Enter authentication details

Provide Service Principal credentials :

* Client ID
* Client Secret

{% hint style="info" %}
Your client secret is encrypted at rest and never exposed to end users. OneLens only accesses billing and metadata tables. We do not access query results or customer datasets.
{% endhint %}

{% endstep %}

{% step %}

<figure><img src="/files/KbONclEelQVxripqulcN" alt=""><figcaption></figcaption></figure>

### Validate Connection

OneLens will automatically verify:

* Authentication
* SQL Warehouse access
* System table permissions
* Ability to query usage data

Once all checks pass, click **Connect**.

<figure><img src="/files/dQ58wdvvSw9mrTxvZUXN" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

### Data handling and security

* OneLens reads only billing and usage metadata from Databricks system tables
* No access to query results or user data
* Credentials are encrypted at rest

***

### Data availability

* Data ingestion begins immediately after connection
* Historical data is imported based on availability in system tables
* Data is refreshed continuously

***

### Troubleshooting

<details>

<summary>Authentication failed</summary>

Verify the Client ID and Client Secret

</details>

<details>

<summary>Warehouse access denied</summary>

Ensure the Service Principal has `CAN USE` permission

</details>

<details>

<summary>Missing system table permissions</summary>

Ensure Data Reader access is granted on all required schemas

</details>

<details>

<summary>No data available</summary>

Confirm Unity Catalog is enabled and system tables contain data

</details>


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.onelens.cloud/integrations/cloud-and-cost-sources/connecting-to-databricks.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.