# Connecting to Databricks

## Connect to Databricks

Connect your Databricks account to OneLens to import usage and billing data for cost analysis, anomaly detection, and optimization insights.

{% hint style="info" %}

#### Before you begin

Ensure the following:

* You have **Account Admin** access in Databricks
* Your workspace is **Unity Catalog-enabled**
* You can create Service Principals and SQL Warehouses
  {% endhint %}

{% hint style="info" %}

#### Required Permissions

The Service Principal used for this integration must have the following permissions:

#### SQL Warehouse

* `CAN USE` on the SQL Warehouse

#### Unity Catalog (System Tables)

Grant **Data Reader** access on:

* `system.billing`
* `system.compute`
* `system.access`
  {% endhint %}

***

## Configuring Databricks

Complete the following steps in your Databricks account before integrating with OneLens

{% stepper %}
{% step %}

### Collect account and workspace details

* Log in to the Databricks console
* Copy your **Databricks Account ID**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2F3alxaqcMgg46M4i1K3y7%2Fimage.png?alt=media&#x26;token=955cc2ee-d59b-4149-b822-e10700446d49" alt=""><figcaption></figcaption></figure>

* Navigate to **Workspaces** and select your workspace
* Copy the **Workspace URL**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FhxXfshIehgHJ7cwUhilF%2Fimage.png?alt=media&#x26;token=1e472cd4-9221-422d-87ff-1307a2d7d7a4" alt=""><figcaption></figcaption></figure>

* Open the workspace
  {% endstep %}

{% step %}

### Create a Service Principal

* Go to: **Settings → Identity and access → Service principals**
* Click **Add service principal**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FrG5xLSYEiFs1kObPA3o0%2Fimage.png?alt=media&#x26;token=5e8b94b6-e0f4-4687-9f06-864dbc3c7a74" alt=""><figcaption></figcaption></figure>

* Enter a name (for example, `onelens-integration-sp`) & click on add
* Open the Service Principal and go to the **Secrets** tab
* Click **Generate secret. When the secret expires, you’ll need to create a new one and reconfigure integration with OneLens.**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FbdaliT8IyOMxHYHGCTxX%2Fimage.png?alt=media&#x26;token=9ad90dad-b6f1-4aeb-90ea-bbc44eca22b0" alt=""><figcaption></figcaption></figure>

* Copy the following and save to use in OneLens :
  * Client ID
  * Client Secret

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FhNwUEvRCB4X53KPLRGGC%2Fimage.png?alt=media&#x26;token=eb318697-f5ec-4b8e-9bce-091b7c4ad037" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Create a SQL Warehouse

* Navigate to: **SQL → SQL Warehouses**
* Click **Create warehouse**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FdvcW6i66Ct8cejlVvhiV%2Fimage.png?alt=media&#x26;token=fcea1d32-cafa-4eab-a0ea-d7e3f4552e62" alt=""><figcaption></figcaption></figure>

* Configure:
  * Type: Serverless
  * Size: **2X-Small**
* Create the warehouse
* Copy the **SQL Warehouse ID**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FfOD0Sid9E6gs3ajPJUNm%2Fimage.png?alt=media&#x26;token=dd6d2ec1-4969-4f25-8272-5df5b7d574a9" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Grant warehouse access

* Open the SQL Warehouse
* Go to **Permissions**
* Add the Service Principal
* Grant `CAN USE` permission

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2F6z2HHaSp2YNlkOysWnPo%2Fimage.png?alt=media&#x26;token=7fffc4b4-1a44-41f1-96eb-71102a3985e7" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Grant system table access

* Navigate to **Catalog → system**
* Grant **Data Reader** access to the Service Principal on:
  * `system.billing`
  * `system.compute`
  * `system.access`

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FtSW48S2HFBhFKyKk2UmN%2Fimage.png?alt=media&#x26;token=f2787d0b-1665-4e79-a7e3-20da7d00c623" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

## Connecting Databricks in OneLens

Complete the following steps in OneLens.

{% stepper %}
{% step %}

### Open the integration flow

1. Navigate to **OneLens → Integrations → Databricks**
2. Click **Connect Databricks**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FHq6xWn9yA3M8UM5jEnjE%2Fimage.png?alt=media&#x26;token=eab1b898-338a-4d14-93d4-bea8ec3cdef3" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Enter account details

1. **Integration Name**
2. **Databricks Account ID**
3. Workspace URL
4. **SQL Warehouse ID**

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FSN3CXZ5aVl6WalV7SkLD%2Fimage.png?alt=media&#x26;token=6347d27c-ea7a-476c-a05e-97d5510e09b3" alt=""><figcaption></figcaption></figure>
{% endstep %}

{% step %}

### Enter authentication details

Provide Service Principal credentials :

* Client ID
* Client Secret

{% hint style="info" %}
Your client secret is encrypted at rest and never exposed to end users. OneLens only accesses billing and metadata tables. We do not access query results or customer datasets.
{% endhint %}

{% endstep %}

{% step %}

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FNEaWH1eP3VFEzwcniYre%2Fimage.png?alt=media&#x26;token=16875dac-d21c-4d7a-8fac-0ebc3d1bfb15" alt=""><figcaption></figcaption></figure>

### Validate Connection

OneLens will automatically verify:

* Authentication
* SQL Warehouse access
* System table permissions
* Ability to query usage data

Once all checks pass, click **Connect**.

<figure><img src="https://3963693991-files.gitbook.io/~/files/v0/b/gitbook-x-prod.appspot.com/o/spaces%2FiyNGpqVYfmDF6qt7Lzar%2Fuploads%2FguIPva6x1ItfoL6rCluk%2Fimage.png?alt=media&#x26;token=b26430a7-e723-4a08-9434-6eab7e923562" alt=""><figcaption></figcaption></figure>
{% endstep %}
{% endstepper %}

***

### Data handling and security

* OneLens reads only billing and usage metadata from Databricks system tables
* No access to query results or user data
* Credentials are encrypted at rest

***

### Data availability

* Data ingestion begins immediately after connection
* Historical data is imported based on availability in system tables
* Data is refreshed continuously

***

### Troubleshooting

<details>

<summary>Authentication failed</summary>

Verify the Client ID and Client Secret

</details>

<details>

<summary>Warehouse access denied</summary>

Ensure the Service Principal has `CAN USE` permission

</details>

<details>

<summary>Missing system table permissions</summary>

Ensure Data Reader access is granted on all required schemas

</details>

<details>

<summary>No data available</summary>

Confirm Unity Catalog is enabled and system tables contain data

</details>