> For the complete documentation index, see [llms.txt](https://docs.onelens.cloud/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.onelens.cloud/integrations/ai-integrations/litellm-integration.md).
# LiteLLM Integration
## TL;DR
* **What this does:** Pulls per-user, per-model token usage and cost data from your LiteLLM Proxy into OneLens — giving you unified cost visibility across every model and provider behind LiteLLM.
* **Time required:** \~10 minutes.
* **Who you need:** The LiteLLM Proxy admin (someone with access to LITELLM\_MASTER\_KEY). One engineer to validate the data.
* **What OneLens reads:** Read-only usage and cost metadata from the LiteLLM Proxy REST API (`/user/daily/activity`, `/user/info`, `/spend/logs`, `/team/list`). Your prompts, completions, and production data are never accessed.
## What You'll Get Once Connected
| Capability | What it does for you |
| ------------------------------------ | -------------------------------------------------------------------------------------------------------------------- |
| **Unified AI Cost Explorer** | View spend across every LLM provider behind LiteLLM (OpenAI, Anthropic, Azure, Bedrock, Groq, etc.) in one dashboard |
| **Per-User Cost Tracking** | See exactly what each user spends, broken down by model and provider |
| **Per-Model Cost Breakdown** | Compare spend across models to find the most cost-effective option for each use case |
| **Per-Provider Spend** | Track spend by provider (OpenAI vs. Anthropic vs. Azure, etc.) to inform contract and commitment decisions |
| **Token Usage Analytics** | Monitor prompt and completion token volumes per model, user, and API key |
| **Multi-Provider Anomaly Detection** | Spot unexpected cost spikes across any provider routed through LiteLLM |
| **Budget Tracking** | Set and monitor budgets at the user or team level with OneLens alerts |
## Security at a Glance
| Question | Answer |
| --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Does OneLens read our prompts or completions? | No. The endpoints returns only aggregated cost and token-count metadata. No prompt or completion content is included. |
| Does OneLens access our LiteLLM database? | No. OneLens connects only via the LiteLLM Proxy REST API. No database credentials are shared. |
| Is the access read-only? | Yes. It cannot generate keys, modify configuration, create models, or invoke any write endpoint. |
| What authentication is used? | The LITELLM\_MASTER\_KEY is used for API access. See the Connectivity Options section for two approaches to limit exposure. |
| Does OneLens see user identifiers? | Yes. The endpoint returns `user_id` values (the internal LiteLLM user). If your users are identified by email or name, these will appear. Use opaque IDs if this is a concern. |
| How is data transmitted and stored? | All API calls use HTTPS/TLS 1.2+. Credentials are encrypted at rest in OneLens using GCP KMS. |
| What is the data retention policy? | OneLens retains ingested data for 12 months by default (configurable). On disconnection or deletion request, data is purged within 30 days with confirmation. |
| Can I restrict by IP? | Yes. If your proxy is behind a firewall, allowlist OneLens's egress IPs or use the push-based approach. |
| Where does the data live? | OneLens infrastructure runs on GCP. See the OneLens Trust & Security page for region details, SOC 2 report, and DPA. |
## Cost of the Integration
OneLens does not create any infrastructure in your environment. All data is pulled via lightweight API calls to your existing LiteLLM Proxy.
| Item | What it is | Typical cost |
| ----------------------- | --------------------------------------------------------------------------------- | ------------ |
| LiteLLM license | Open-source (MIT) — free | $0 |
| Compute on your side | OneLens calls the LiteLLM Proxy API; your proxy serves from its existing database | $0 |
| Data egress | Daily activity responses are small JSON payloads — typically <100 KB/day | <$0.01/month |
| Storage in your account | None — OneLens stores the data in its own infrastructure | $0 |
| **Estimated total** | | **$0/month** |
## How It Works
LiteLLM Proxy sits between your applications and LLM providers. Every request is logged with cost, token counts, model, provider, user, and API key. LiteLLM computes cost per request using its model pricing map.
OneLens pulls from multiple read-only endpoints mentioned in [#what-onelens-will-access](#what-onelens-will-access "mention") on your proxy daily.
## Connectivity Options
Since you're running open-source LiteLLM, the scoped `get_spend_routes` virtual key (an Enterprise feature) is not available. OneLens authenticates using the LITELLM\_MASTER\_KEY. Here are two approaches to manage this securely:
### Option A: Master Key Direct (Recommended)
| Aspect | Detail |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **How it works** | Share the LITELLM\_MASTER\_KEY with OneLens. OneLens calls the apis mentioned in in [#what-onelens-will-access](#what-onelens-will-access "mention") on your proxy once per day. |
| **Security** | OneLens contractually commits to calling only read endpoints. An immutable audit log of every API call is maintained and available on request. OneLens never calls write/admin endpoints. |
| **Setup complexity** | Low — just provide the key and proxy URL. Done in 2 minutes. |
| **Best for** | Most teams. Quickest path to value with contractual security controls. |
### Option B: API Gateway (For Additional Isolation)
| Aspect | Detail |
| -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **How it works** | Place nginx, Kong, or AWS ALB in front of your proxy. Allowlist only the apis mentioned in [#what-onelens-will-access](#what-onelens-will-access "mention") for OneLens. The master key is injected server-side — OneLens never sees it. |
| **Security** | Stronger. Master key never leaves your network. OneLens only reaches one whitelisted endpoint. |
| **Setup complexity** | Moderate — requires API gateway config (see nginx example below). |
| **Best for** | Teams with existing API gateway infrastructure who want an extra layer of isolation. |
### Option B: Nginx Example
```nginx
# /etc/nginx/conf.d/litellm-onelens.conf
location ~ ^/(user/daily/activity|user/info|spend/logs|team/list) {
allow 35.244.46.16/32; # OneLens IP 1
allow 34.120.137.88/32; # OneLens IP 2
allow 35.201.115.143/32; # OneLens IP 3
deny all;
limit_except GET { deny all; }
proxy_pass http://localhost:4000;
proxy_set_header Authorization "Bearer YOUR_LITELLM_MASTER_KEY_HERE";
}
```
With this setup, you share the nginx endpoint URL with OneLens (not the master key). OneLens authenticates to nginx via IP allowlist, and nginx injects the master key when forwarding to LiteLLM.
> ***Our recommendation:** Option A is the fastest path — provide the key, connect, and you're done. If your security team requires that the master key not leave your network, use Option B with the nginx config below.*
## What OneLens Will Access
| Endpoint | Purpose | OSS / Enterprise |
| -------------------------- | --------------------------------------------------------------------------- | ------------------- |
| *GET /user/daily/activity* | Per-user daily breakdown by model, provider, and API key (primary endpoint) | OSS |
| *GET /user/info* | Total spend per user, their keys, and team memberships | OSS |
| *GET /team/list* | List all teams with spend and budget data | OSS |
| *GET /spend/logs* | Individual transaction logs with model, tokens, cost per request | OSS |
| *GET /global/spend/report* | Aggregated spend reports grouped by team, customer, or user | **Enterprise only** |
> **Note:** OneLens works fully with open-source LiteLLM using the OSS endpoints above. The `/global/spend/report` endpoint is only available on LiteLLM Enterprise — if your deployment uses OSS LiteLLM, OneLens will derive equivalent reports from `/user/daily/activity` and `/spend/logs`.
## What OneLens Will NOT Access
* `/chat/completions`, `/completions`, `/embeddings` — OneLens cannot invoke any model
* `/key/generate`, `/key/update`, `/key/delete` — OneLens cannot create, modify, or delete API keys
* `/model/new`, `/model/update`, `/model/delete` — OneLens cannot modify model configuration
* `/config/*` — OneLens cannot read or modify proxy configuration
* The LiteLLM PostgreSQL database — OneLens connects only via the REST API
* Any prompt or completion content — the endpoints above return only aggregated metadata
## Prerequisites
* LiteLLM Proxy running with a connected PostgreSQL database and spend tracking enabled.
* `LITELLM_MASTER_KEY` — needed for API access (or the nginx gateway URL if using Option A).
* Network access: OneLens must be able to reach your proxy (or gateway) over HTTPS.
**What OneLens will access:**
* `GET /user/daily/activity` — daily aggregated usage per user (cost, tokens, request count, broken down by model, provider, API key)
**What OneLens will NOT access:**
* Prompt or completion content
* The LiteLLM database directly
* Any write, mutate, or admin endpoint (`/key/generate`, `/model/new`, `/global/spend/reset`, etc.)
* Your LLM provider credentials
### Validate and Connect
{% stepper %}
{% step %}
#### Step 1: Validate the Endpoint
Run this from your terminal to confirm the endpoint works and returns useful data:
```bash
curl -X GET \
'https://your-litellm-proxy:4000/user/daily/activity' \
-G -d 'start_date=2025-01-01' \
-d 'end_date=2025-01-07' \
-H 'Authorization: Bearer YOUR_LITELLM_MASTER_KEY_HERE'
```
Expected response structure:
```json
{
"results": [
{
"date": "2025-01-07",
"metrics": {
"spend": 0.0177,
"prompt_tokens": 111,
"completion_tokens": 1711,
"total_tokens": 1822,
"api_requests": 11
},
"breakdown": {
"models": { "gpt-4o-mini": { "spend": 0.001, ... } },
"providers": { "openai": { ... }, "anthropic": { ... } },
"api_keys": { "3126b6ea...": { ... } }
}
}
],
"metadata": {
"total_spend": 0.727,
"total_prompt_tokens": 280990,
"total_completion_tokens": 376674,
"total_api_requests": 14
}
}
```
> **Tip:** If the response is empty, check that your proxy has recent LLM traffic and spend tracking is enabled (requires a connected PostgreSQL database). If this returns 401 or 403, verify your master key is correct. If it returns an empty results array, check that your proxy has recent LLM traffic with spend tracking enabled.
> {% endstep %}
{% step %}
#### Step 2: Validate Data Richness
Inspect the response from Step 1 for the following:
| Field | What to check |
| ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Per-user data | The response should show activity for individual users. If all activity is under one user, verify that API keys are being generated with `user_id` set. |
| Model breakdown | `breakdown.models` should show your active models (e.g., `gpt-4o`, `claude-sonnet-4-20250514`). If missing, check your `config.yaml` model list. |
| Provider breakdown | `breakdown.providers` should show multiple providers if you route to more than one. |
| Non-zero spend | `metrics.spend` should be >0. If $0.00, the model may not be in LiteLLM's pricing map — add custom pricing or enable `sync_model_pricing_from_github: true`. |
| Token counts | `prompt_tokens` and `completion_tokens` should be non-zero. If zero, check your LiteLLM version. |
If spend is $0 for known models: LiteLLM uses a community pricing map. Ensure it's current by adding `sync_model_pricing_from_github: true` under `litellm_settings` in your `config.yaml`. For negotiated rates, set `input_cost_per_token` and `output_cost_per_token` per model. See Custom LLM Pricing.
{% endstep %}
{% step %}
#### Step 3: Connect to OneLens
Provide the following to OneLens:
| Field | Value | Example |
| ------------------------------ | -------------------------------------------------------------- | ----------------------------------------- |
| LiteLLM endpoint | Your proxy URL (or nginx gateway URL if using Option A) | `https://litellm-gateway.yourcompany.com` |
| Authentication | Master key (Option B/C) or none if nginx injects it (Option A) | `sk-your-master-key` |
| Polling interval | How often OneLens should pull data (default: daily) | `daily` |
| Historical backfill start date | Earliest date to pull data from | `2025-01-01` |
OneLens will begin ingesting data on the next scheduled poll.
> **Secure key sharing:** Never share your LITELLM\_MASTER\_KEY over email or chat. Use a validated secure sharing tool like [Password.link](https://password.link/en) to transmit credentials safely.
> {% endstep %}
> {% endstepper %}
## Data Refresh Schedule
OneLens polls your LiteLLM Proxy once daily by default.
1. LiteLLM records spend in real-time as each LLM request completes — the `/user/daily/activity` endpoint reflects data aggregated by day.
2. Daily polling is sufficient because this endpoint provides daily aggregates — polling more frequently doesn't yield fresher data.
3. Current-day figures may shift as in-progress requests complete and provider-specific adjustments are applied.
## Data Privacy & Security
* **API-only** — no database credentials, no SSH, no agent installed in your environment.
* **No prompt/completion content** — the endpoint returns only aggregated cost and token metadata.
* **TLS in transit** — all API calls use HTTPS/TLS 1.2+.
* **Encrypted at rest** — credentials and ingested data are encrypted at rest using GCP KMS in OneLens.
* **Network policy support** — restrict access via API gateway (Option B) or allowlist OneLens egress IPs.
* **Data retention** — 12-month default, configurable. Deletion within 30 days on request with confirmation.
* **User ID exposure** — `user_id` values from LiteLLM are ingested when set on API keys. API keys without a `user_id` are still tracked at the key level but cannot be attributed to a specific user. To enable per-user attribution, ensure `user_id` is set when generating keys. Use opaque IDs if PII is a concern.
## Troubleshooting
| Symptom | Cause | Fix |
| ----------------------------------- | ------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------- |
| 401 Unauthorized | Invalid master key | Verify `LITELLM_MASTER_KEY` value. Check `Authorization: Bearer` header format. |
| Empty results array | No LLM traffic in the date range, or spend tracking not enabled | Widen the date range. Verify your proxy has a connected PostgreSQL database. |
| All activity under one user | API keys not assigned to individual users | Generate keys with `user_id` parameter: `/key/generate -d '{"user_id": "jane"}'`. |
| Spend is $0 for known models | Model not in LiteLLM's pricing map | > **If spend is $0 for known models:** Enable `sync_model_pricing_from_github: true` or set custom pricing per model. |
| Missing model in breakdown | Model not in your `config.yaml` model list | Add the model to your LiteLLM `config.yaml` and restart the proxy. |
| Cost doesn't match provider invoice | LiteLLM uses community pricing; negotiated/committed rates not reflected | Configure actual rates via Custom LLM Pricing. |
| No data in OneLens after 24 hours | Proxy unreachable, key wrong, or firewall blocking | Run Step 1 manually. Check proxy access logs. |
| Nginx gateway returns 502 | LiteLLM proxy is down or unreachable from nginx | Check proxy health: `curl http://localhost:4000/health`. |
## Frequently Asked Questions
Can OneLens see our prompts or completions?
No. The `/user/daily/activity` endpoint returns only aggregated daily metrics — spend, token counts, request counts — broken down by model, provider, and API key. No request or response content is included.
Is the master key safe to share?
> **Security note:** The master key has full admin access to your LiteLLM Proxy. For maximum isolation, use Option B (API gateway) so the key never leaves your network.
The master key has full admin access to your LiteLLM Proxy. OneLens contractually commits to read-only usage and maintains audit logs. If your security team requires the master key not leave your network, use Option B (API gateway) — the master key stays inside your network and OneLens only reaches a single whitelisted endpoint.
Will this impact our proxy's performance?
No. OneLens makes one GET request per day. The `/user/daily/activity` endpoint serves pre-aggregated data — it's a lightweight query.
Can I connect multiple LiteLLM Proxy instances?
Yes. Add each as a separate connection in OneLens with a source label (e.g., `prod`, `staging`).
How do I disconnect OneLens?
Rotate the master key (Option A) or delete the nginx gateway config (Option B). OneLens stops receiving data immediately. Historical data remains until you request deletion.
What if cost doesn't match my provider invoices?
LiteLLM computes cost using its community pricing map. If you have negotiated discounts or committed-use pricing (e.g., Bedrock Provisioned Throughput), configure your actual rates in LiteLLM via Custom LLM Pricing. Committed-capacity pricing is fundamentally per-hour, not per-token — OneLens shows the on-demand equivalent, which is useful for optimization but won't match the committed invoice.
Can we upgrade to scoped keys later?
Yes. If you move to LiteLLM Enterprise, you can generate a key with `permissions: {"get_spend_routes": true}` — this key can only read spend data and nothing else. The doc can be updated to use that approach at that point.
***
## Need Help?
**LiteLLM official docs:**
* [Spend Tracking — how LiteLLM tracks cost by key, user, team](https://docs.litellm.ai/docs/proxy/cost_tracking)
* [Virtual Keys & Authentication — key generation, permissions, database setup](https://docs.litellm.ai/docs/proxy/virtual_keys)
* [Custom LLM Pricing — configuring negotiated rates](https://docs.litellm.ai/docs/proxy/custom_pricing)
* [Daily Spend Breakdown API — Swagger reference](https://litellm-api.up.railway.app/#/Budget%20%26%20Spend%20Tracking/get_user_daily_activity_user_daily_act)
**OneLens support:**
---
# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.
## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:
```
GET https://docs.onelens.cloud/integrations/ai-integrations/litellm-integration.md?ask=&goal=
```
`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.