> For the complete documentation index, see [llms.txt](https://docs.onelens.cloud/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.onelens.cloud/optimize-cost-savings-and-recommendations/ai-cost-optimization.md). # AI Cost Optimization > Reduce AI costs without compromising application quality or user experience. AI Cost Optimization continuously analyzes your AI workloads and identifies opportunities to reduce spend across models, tokens, caching, infrastructure, and usage patterns. Instead of generic recommendations, OneLens provides actionable insights with estimated savings and clear remediation guidance. *** ### Why AI Cost Optimization? AI costs grow quickly as applications scale. Common reasons include: * Using larger models than necessary * Inefficient prompts * Poor cache utilization * Excessive output tokens * Idle provisioned throughput * Batch-eligible workloads running synchronously * Outdated model versions * Over-provisioned infrastructure OneLens helps teams identify these opportunities early and optimize AI spend continuously. *** ### Optimization Categories Recommendations are grouped into logical categories, making it easier to prioritize improvements. * Token Optimization * Model Optimization * Infrastructure Optimization * Workload Optimization
*** ## Token Optimization Reduce costs by improving how tokens are consumed. Recommendations include: #### Prompt Cache Optimization Identify workloads with low cache hit rates and opportunities to improve prompt reuse. *** #### Output Token Optimization Detect applications generating unnecessarily long responses that increase costs. Identify: * Verbose prompts * Excessive completion lengths * High output-to-input ratios *** #### Batch API Opportunities Find workloads suitable for asynchronous batch processing to take advantage of lower pricing where supported. Ideal for: * Document processing * Report generation * Background workflows * Bulk inference jobs *** ## Model Optimization Ensure every workload uses the most cost-effective model. #### Model Right-Sizing Identify workloads where a smaller or less expensive model can deliver similar results. Examples: * GPT-4 → GPT-4o Mini * Claude Opus → Claude Sonnet * Large embedding model → Smaller embedding model *** #### Model Version Management Detect deprecated or older model versions still running in production. Stay updated with supported models while improving cost efficiency. *** #### Provisioned vs On-Demand Usage Analyze traffic patterns to determine whether workloads would benefit from: * On-Demand * Flex * Provisioned Throughput This helps optimize both cost and performance. *** ## Infrastructure Optimization Optimize the infrastructure supporting AI workloads. #### Idle Fine-Tuned Models Identify fine-tuned models with little or no inference traffic. Remove unused resources and reduce operational costs. *** #### Endpoint Right-Sizing Analyze throughput and request volume to recommend appropriately sized inference endpoints. *** #### Reserved Capacity Utilization Detect underutilized reserved or provisioned capacity and identify opportunities to improve utilization. *** ## Workload Optimization Recommendations tailored to different AI workloads. #### Conversational AI Improve: * Cache utilization * Model selection * Output length * Session efficiency *** #### Agentic Workflows Optimize: * Recursive agent loops * Multi-step execution * Model routing * Token consumption per workflow *** #### RAG Applications Identify opportunities to improve: * Embedding model selection * Chunk sizing * Context length * Retrieval efficiency *** #### Coding Assistants Optimize: * Model selection * Batch execution * Prompt reuse * Cache effectiveness *** #### Document Processing Identify batch-friendly workloads and reduce costs through asynchronous processing and optimized model selection. *** ### Estimated Savings Every recommendation includes an estimate of its potential impact. View: * Monthly savings * Annual savings * Percentage reduction * Recommendation priority * Expected effort This helps teams focus on changes that deliver the highest return. *** ### Prioritize Recommendations Recommendations can be filtered by: * Estimated savings * Provider * Team * Project * Environment * Category * Priority This makes it easier to plan optimization initiatives across multiple teams. *** ### Track Optimization Progress Monitor the status of every recommendation throughout its lifecycle. Typical statuses include: * Open * Planned * In Progress * Completed * Ignored This provides visibility into realized savings and ongoing optimization efforts. > 📸 **Screenshot:** Recommendation details *** ### Example Use Cases #### Engineering Reduce AI costs by selecting the right models, optimizing prompts, and improving cache efficiency. #### Platform Teams Optimize provisioned throughput, inference endpoints, and infrastructure utilization. #### Product Teams Deliver AI features at lower operating costs without affecting user experience. #### Finance & FinOps Track realized savings and measure the financial impact of optimization initiatives. *** ### Benefits * Reduce AI spend without sacrificing quality. * Identify optimization opportunities automatically. * Prioritize recommendations based on business impact. * Improve model and infrastructure efficiency. * Continuously monitor AI cost optimization opportunities. *** ### Best Practices * Review optimization recommendations regularly instead of waiting for monthly invoices. * Prioritize high-impact recommendations with minimal implementation effort. * Validate model right-sizing before rolling changes to production. * Combine optimization insights with anomaly detection and budgets for proactive cost management. * Track completed recommendations to measure realized savings over time. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.onelens.cloud/optimize-cost-savings-and-recommendations/ai-cost-optimization.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.