AI Cost Optimization

AI Cost Optimization — ɳSelf documentation.

The AI plugin includes a built-in cost optimization engine (AA09) that analyzes your request history and automatically routes tasks to cheaper models when quality is not at risk.

How It Works

Task classification: each request is classified into one of eight task types using lightweight keyword heuristics (no extra API call required for common cases).
Retroactive analysis: a nightly job (or on-demand via nself ai cost report) reads np_ai_requests from the last 30 days and computes what each request would have cost on cheaper models.
Recommendations: the engine produces per-task-class recommendations showing current cost, projected cost, savings percentage, and estimated quality risk.
Routing rules: you can apply a recommendation as a routing rule with one click in the admin dashboard or via CLI. Subsequent requests of that class are sent to the cheaper model automatically.
Quality sampling: 5% of routed requests are still sent to the original model for comparison. If quality delta exceeds 0.15 (cosine similarity), the rule is flagged for review.

Task Classes

Class	Description	Can use cheaper model
`code_generation`	Write functions, classes, scripts	No
`summarization`	TL;DR, bullet points, key points	Yes
`qa_factual`	Simple factual questions	Yes
`qa_reasoning`	Multi-step logic, math, proofs	No
`translation`	Language translation	Yes
`chat_casual`	Casual conversation	Yes
`structured_extraction`	Parse JSON, CSV, tables	Yes
`analysis_complex`	Evaluate, compare, research	No

CLI Commands

# View cost analysis for the last 30 days
nself ai cost report

# Specify a time window
nself ai cost report --last 30d

# Show projected monthly savings if all recommendations applied
nself ai cost savings

# Apply a routing rule from a recommendation
nself ai cost apply <rule-id>

# List active routing rules
nself ai cost rules list

# Disable a routing rule
nself ai cost rules disable <id>

# Run a quality benchmark for a task class
nself ai cost benchmark <class>

Admin Dashboard

Navigate to AI > Cost Optimization in the admin panel to:

See your Optimization Score (0–100): how much of the available savings you are currently capturing.
Review the top 5 recommendations with one-click Apply rule buttons.
Manage active routing rules (disable any rule that degrades quality).

Model Tier Order

Requests are routed from cheapest to most capable based on task class:

Haiku / Flash / GPT-4o-mini  (cheap)
Sonnet / Flash               (mid)
Opus / GPT-4o / Pro          (capable)

Local Ollama models are always $0 and are used when available and configured.

Database Tables

Table	Purpose
`np_ai_optimization_recs`	Generated recommendations (upserted by each analysis run)
`np_ai_routing_rules`	User-applied model override rules
`np_ai_quality_samples`	5% quality sampling results
`np_ai_requests.task_class`	Per-request task class label (added by AA09 migration)

Quality Risk Levels

Low: summarization, translation, factual QA on a medium-tier model. Well-studied; cost savings reliable.
Medium: structured extraction on the cheapest model tier. Generally fine; spot-check outputs.
High: code generation or complex reasoning on a cheaper model. Not recommended; rule creation is blocked for these classes.