LiteLLM Integration

Connecting LiteLLM Proxy spend and usage data to OneLens for unified AI cost management.

TL;DR

• What this does: Pulls per-user, per-model token usage and cost data from your LiteLLM Proxy into OneLens — giving you unified cost visibility across every model and provider behind LiteLLM.

• Time required: ~10 minutes.

• Who you need: The LiteLLM Proxy admin (someone with access to LITELLM_MASTER_KEY). One engineer to validate the data.

• What OneLens reads: Read-only usage and cost metadata from the LiteLLM Proxy REST API (/user/daily/activity, /user/info, /spend/logs, /team/list). Your prompts, completions, and production data are never accessed.

What You'll Get Once Connected

Capability	What it does for you
Unified AI Cost Explorer	View spend across every LLM provider behind LiteLLM (OpenAI, Anthropic, Azure, Bedrock, Groq, etc.) in one dashboard
Per-User Cost Tracking	See exactly what each user spends, broken down by model and provider
Per-Model Cost Breakdown	Compare spend across models to find the most cost-effective option for each use case
Per-Provider Spend	Track spend by provider (OpenAI vs. Anthropic vs. Azure, etc.) to inform contract and commitment decisions
Token Usage Analytics	Monitor prompt and completion token volumes per model, user, and API key
Multi-Provider Anomaly Detection	Spot unexpected cost spikes across any provider routed through LiteLLM
Budget Tracking	Set and monitor budgets at the user or team level with OneLens alerts

Security at a Glance

Question	Answer
Does OneLens read our prompts or completions?	No. The endpoints returns only aggregated cost and token-count metadata. No prompt or completion content is included.
Does OneLens access our LiteLLM database?	No. OneLens connects only via the LiteLLM Proxy REST API. No database credentials are shared.
Is the access read-only?	Yes. It cannot generate keys, modify configuration, create models, or invoke any write endpoint.
What authentication is used?	The LITELLM_MASTER_KEY is used for API access. See the Connectivity Options section for two approaches to limit exposure.
Does OneLens see user identifiers?	Yes. The endpoint returns user_id values (the internal LiteLLM user). If your users are identified by email or name, these will appear. Use opaque IDs if this is a concern.
How is data transmitted and stored?	All API calls use HTTPS/TLS 1.2+. Credentials are encrypted at rest in OneLens using GCP KMS.
What is the data retention policy?	OneLens retains ingested data for 12 months by default (configurable). On disconnection or deletion request, data is purged within 30 days with confirmation.
Can I restrict by IP?	Yes. If your proxy is behind a firewall, allowlist OneLens's egress IPs or use the push-based approach.
Where does the data live?	OneLens infrastructure runs on GCP. See the OneLens Trust & Security page for region details, SOC 2 report, and DPA.

Cost of the Integration

OneLens does not create any infrastructure in your environment. All data is pulled via lightweight API calls to your existing LiteLLM Proxy.

Item	What it is	Typical cost
LiteLLM license	Open-source (MIT) — free	$0
Compute on your side	OneLens calls the LiteLLM Proxy API; your proxy serves from its existing database	$0
Data egress	Daily activity responses are small JSON payloads — typically <100 KB/day	<$0.01/month
Storage in your account	None — OneLens stores the data in its own infrastructure	$0
Estimated total		$0/month

How It Works

LiteLLM Proxy sits between your applications and LLM providers. Every request is logged with cost, token counts, model, provider, user, and API key. LiteLLM computes cost per request using its model pricing map.

OneLens pulls from multiple read-only endpoints mentioned in What OneLens Will Access on your proxy daily.

Connectivity Options

Since you're running open-source LiteLLM, the scoped get_spend_routes virtual key (an Enterprise feature) is not available. OneLens authenticates using the LITELLM_MASTER_KEY. Here are two approaches to manage this securely:

Option A: Master Key Direct (Recommended)

Aspect	Detail
How it works	Share the LITELLM_MASTER_KEY with OneLens. OneLens calls the apis mentioned in in What OneLens Will Access on your proxy once per day.
Security	OneLens contractually commits to calling only read endpoints. An immutable audit log of every API call is maintained and available on request. OneLens never calls write/admin endpoints.
Setup complexity	Low — just provide the key and proxy URL. Done in 2 minutes.
Best for	Most teams. Quickest path to value with contractual security controls.

Option B: API Gateway (For Additional Isolation)

Aspect	Detail
How it works	Place nginx, Kong, or AWS ALB in front of your proxy. Allowlist only the apis mentioned in What OneLens Will Access for OneLens. The master key is injected server-side — OneLens never sees it.
Security	Stronger. Master key never leaves your network. OneLens only reaches one whitelisted endpoint.
Setup complexity	Moderate — requires API gateway config (see nginx example below).
Best for	Teams with existing API gateway infrastructure who want an extra layer of isolation.

Option B: Nginx Example

# /etc/nginx/conf.d/litellm-onelens.conf
location ~ ^/(user/daily/activity|user/info|spend/logs|team/list) {
    allow 35.244.46.16/32;   # OneLens IP 1
    allow 34.120.137.88/32;  # OneLens IP 2
    allow 35.201.115.143/32; # OneLens IP 3
    deny all;
    limit_except GET { deny all; }
    proxy_pass http://localhost:4000;
    proxy_set_header Authorization "Bearer YOUR_LITELLM_MASTER_KEY_HERE";
}

With this setup, you share the nginx endpoint URL with OneLens (not the master key). OneLens authenticates to nginx via IP allowlist, and nginx injects the master key when forwarding to LiteLLM.

Our recommendation: Option A is the fastest path — provide the key, connect, and you're done. If your security team requires that the master key not leave your network, use Option B with the nginx config below.

What OneLens Will Access

Endpoint	Purpose	OSS / Enterprise
GET /user/daily/activity	Per-user daily breakdown by model, provider, and API key (primary endpoint)	OSS
GET /user/info	Total spend per user, their keys, and team memberships	OSS
GET /team/list	List all teams with spend and budget data	OSS
GET /spend/logs	Individual transaction logs with model, tokens, cost per request	OSS
GET /global/spend/report	Aggregated spend reports grouped by team, customer, or user	Enterprise only

Note: OneLens works fully with open-source LiteLLM using the OSS endpoints above. The /global/spend/report endpoint is only available on LiteLLM Enterprise — if your deployment uses OSS LiteLLM, OneLens will derive equivalent reports from /user/daily/activity and /spend/logs.

What OneLens Will NOT Access

• /chat/completions, /completions, /embeddings — OneLens cannot invoke any model

• /key/generate, /key/update, /key/delete — OneLens cannot create, modify, or delete API keys

• /model/new, /model/update, /model/delete — OneLens cannot modify model configuration

• /config/* — OneLens cannot read or modify proxy configuration

• The LiteLLM PostgreSQL database — OneLens connects only via the REST API

• Any prompt or completion content — the endpoints above return only aggregated metadata

Prerequisites

• LiteLLM Proxy running with a connected PostgreSQL database and spend tracking enabled.

• LITELLM_MASTER_KEY — needed for API access (or the nginx gateway URL if using Option A).

• Network access: OneLens must be able to reach your proxy (or gateway) over HTTPS.

What OneLens will access:

• GET /user/daily/activity — daily aggregated usage per user (cost, tokens, request count, broken down by model, provider, API key)

What OneLens will NOT access:

• Prompt or completion content

• The LiteLLM database directly

• Any write, mutate, or admin endpoint (/key/generate, /model/new, /global/spend/reset, etc.)

• Your LLM provider credentials

Validate and Connect

1

Step 1: Validate the Endpoint

Run this from your terminal to confirm the endpoint works and returns useful data:

curl -X GET \
  'https://your-litellm-proxy:4000/user/daily/activity' \
  -G -d 'start_date=2025-01-01' \
     -d 'end_date=2025-01-07' \
  -H 'Authorization: Bearer YOUR_LITELLM_MASTER_KEY_HERE'

Expected response structure:

{
  "results": [
    {
      "date": "2025-01-07",
      "metrics": {
        "spend": 0.0177,
        "prompt_tokens": 111,
        "completion_tokens": 1711,
        "total_tokens": 1822,
        "api_requests": 11
      },
      "breakdown": {
        "models": { "gpt-4o-mini": { "spend": 0.001, ... } },
        "providers": { "openai": { ... }, "anthropic": { ... } },
        "api_keys": { "3126b6ea...": { ... } }
      }
    }
  ],
  "metadata": {
    "total_spend": 0.727,
    "total_prompt_tokens": 280990,
    "total_completion_tokens": 376674,
    "total_api_requests": 14
  }
}

Tip: If the response is empty, check that your proxy has recent LLM traffic and spend tracking is enabled (requires a connected PostgreSQL database). If this returns 401 or 403, verify your master key is correct. If it returns an empty results array, check that your proxy has recent LLM traffic with spend tracking enabled.

2

Step 2: Validate Data Richness

Inspect the response from Step 1 for the following:

Field	What to check
Per-user data	The response should show activity for individual users. If all activity is under one user, verify that API keys are being generated with user_id set.
Model breakdown	breakdown.models should show your active models (e.g., gpt-4o, claude-sonnet-4-20250514). If missing, check your config.yaml model list.
Provider breakdown	breakdown.providers should show multiple providers if you route to more than one.
Non-zero spend	metrics.spend should be >0. If $0.00, the model may not be in LiteLLM's pricing map — add custom pricing or enable sync_model_pricing_from_github: true.
Token counts	prompt_tokens and completion_tokens should be non-zero. If zero, check your LiteLLM version.

If spend is $0 for known models: LiteLLM uses a community pricing map. Ensure it's current by adding sync_model_pricing_from_github: true under litellm_settings in your config.yaml. For negotiated rates, set input_cost_per_token and output_cost_per_token per model. See Custom LLM Pricing.

3

Step 3: Connect to OneLens

Provide the following to OneLens:

Field	Value	Example
LiteLLM endpoint	Your proxy URL (or nginx gateway URL if using Option A)	https://litellm-gateway.yourcompany.com
Authentication	Master key (Option B/C) or none if nginx injects it (Option A)	sk-your-master-key
Polling interval	How often OneLens should pull data (default: daily)	daily
Historical backfill start date	Earliest date to pull data from	2025-01-01

OneLens will begin ingesting data on the next scheduled poll.

Secure key sharing: Never share your LITELLM_MASTER_KEY over email or chat. Use a validated secure sharing tool like Password.link to transmit credentials safely.

Data Refresh Schedule

OneLens polls your LiteLLM Proxy once daily by default.

1. LiteLLM records spend in real-time as each LLM request completes — the /user/daily/activity endpoint reflects data aggregated by day.

2. Daily polling is sufficient because this endpoint provides daily aggregates — polling more frequently doesn't yield fresher data.

3. Current-day figures may shift as in-progress requests complete and provider-specific adjustments are applied.

Data Privacy & Security

• API-only — no database credentials, no SSH, no agent installed in your environment.

• No prompt/completion content — the endpoint returns only aggregated cost and token metadata.

• TLS in transit — all API calls use HTTPS/TLS 1.2+.

• Encrypted at rest — credentials and ingested data are encrypted at rest using GCP KMS in OneLens.

• Network policy support — restrict access via API gateway (Option B) or allowlist OneLens egress IPs.

• Data retention — 12-month default, configurable. Deletion within 30 days on request with confirmation.

• User ID exposure — user_id values from LiteLLM are ingested when set on API keys. API keys without a user_id are still tracked at the key level but cannot be attributed to a specific user. To enable per-user attribution, ensure user_id is set when generating keys. Use opaque IDs if PII is a concern.

Troubleshooting

Symptom	Cause	Fix
401 Unauthorized	Invalid master key	Verify LITELLM_MASTER_KEY value. Check Authorization: Bearer header format.
Empty results array	No LLM traffic in the date range, or spend tracking not enabled	Widen the date range. Verify your proxy has a connected PostgreSQL database.
All activity under one user	API keys not assigned to individual users	Generate keys with user_id parameter: /key/generate -d '{"user_id": "jane"}'.
Spend is $0 for known models	Model not in LiteLLM's pricing map	> If spend is $0 for known models: Enable sync_model_pricing_from_github: true or set custom pricing per model.
Missing model in breakdown	Model not in your config.yaml model list	Add the model to your LiteLLM config.yaml and restart the proxy.
Cost doesn't match provider invoice	LiteLLM uses community pricing; negotiated/committed rates not reflected	Configure actual rates via Custom LLM Pricing.
No data in OneLens after 24 hours	Proxy unreachable, key wrong, or firewall blocking	Run Step 1 manually. Check proxy access logs.
Nginx gateway returns 502	LiteLLM proxy is down or unreachable from nginx	Check proxy health: curl http://localhost:4000/health.

Frequently Asked Questions

Can OneLens see our prompts or completions?

No. The /user/daily/activity endpoint returns only aggregated daily metrics — spend, token counts, request counts — broken down by model, provider, and API key. No request or response content is included.

Is the master key safe to share?

Security note: The master key has full admin access to your LiteLLM Proxy. For maximum isolation, use Option B (API gateway) so the key never leaves your network.

The master key has full admin access to your LiteLLM Proxy. OneLens contractually commits to read-only usage and maintains audit logs. If your security team requires the master key not leave your network, use Option B (API gateway) — the master key stays inside your network and OneLens only reaches a single whitelisted endpoint.

Will this impact our proxy's performance?

No. OneLens makes one GET request per day. The /user/daily/activity endpoint serves pre-aggregated data — it's a lightweight query.

Can I connect multiple LiteLLM Proxy instances?

Yes. Add each as a separate connection in OneLens with a source label (e.g., prod, staging).

How do I disconnect OneLens?

Rotate the master key (Option A) or delete the nginx gateway config (Option B). OneLens stops receiving data immediately. Historical data remains until you request deletion.

What if cost doesn't match my provider invoices?

LiteLLM computes cost using its community pricing map. If you have negotiated discounts or committed-use pricing (e.g., Bedrock Provisioned Throughput), configure your actual rates in LiteLLM via Custom LLM Pricing. Committed-capacity pricing is fundamentally per-hour, not per-token — OneLens shows the on-demand equivalent, which is useful for optimization but won't match the committed invoice.

Can we upgrade to scoped keys later?

Yes. If you move to LiteLLM Enterprise, you can generate a key with permissions: {"get_spend_routes": true} — this key can only read spend data and nothing else. The doc can be updated to use that approach at that point.

---

Need Help?

LiteLLM official docs:

• Spend Tracking — how LiteLLM tracks cost by key, user, team

• Virtual Keys & Authentication — key generation, permissions, database setup

• Custom LLM Pricing — configuring negotiated rates

• Daily Spend Breakdown API — Swagger reference

OneLens support: [e-mail]