AI Cost Monitoring

Last updated: 2026-05-12 Owner: Product Writer Audience: platform owner, super_admin, ops_admin

Purpose

Explain how to use the Workspace › Platform › AI Usage dashboard to monitor AI call costs (LLM, embeddings, reranker) by tenant, model, and call kind.

Prerequisites

• platform role super_admin or permission platform.ai.global_policy.manage
• migration 0071_ai_usage_events applied on the Supabase project (typically done by the platform DBA)

Key concepts

• AI Usage event: every call to AI models (Sonnet, Haiku, OpenAI embedding, Cohere rerank) is recorded into ai_usage_events with tenant, user, session, model, in/out tokens, latency, cost in USD micros, and error flag
• Cost in USD micros: cost is stored as bigint in micros (1 USD = 1,000,000 micros) to avoid floating-point drift at scale
• Non-blocking ingestion: logging is fire-and-forget — if the DB is slow or the table is missing, the chat-hr pipeline keeps working without surfacing errors to the user
• Group by: the dashboard aggregates by day (trend), model (breakdown), tenant (per customer), or kind (chat_main, haiku_off_topic, haiku_entity, embedding, haiku_rerank, cohere_rerank)

What you find in the dashboard

• 10 highest-cost sessions in range
• Columns: session id (truncated), org (truncated), cost, calls

Basic steps

1. Navigate to Workspace → Platform → AI Usage
2. Pick a range (default last 30 days)
3. Set Primary = day for the trend, Secondary = model for the breakdown
4. Review the KPIs at the top for the period's total cost
5. Identify in the breakdown chart the model/tenant/kind with highest cost
6. Open the top sessions table to investigate the most expensive conversations

Tracked call kinds

• chat_main — main Sonnet 4.6 agent inside a chat-hr turn
• haiku_off_topic — AI 02 W02 classifier (Haiku 4.5)
• haiku_entity — AI 02 W03 8-dim entity extractor (Haiku 4.5)
• haiku_summary — AI 01 episodic memory summarizer (Haiku 4.5)
• embedding — OpenAI text-embedding-3-large from AI 02 W01
• haiku_rerank — W04 Sonnet/Haiku reranker
• cohere_rerank — W04 Cohere reranker (scaffold)

Models and pricing (2026 reference)

| Model | Input $/M tok | Output $/M tok | Notes | |---|---|---|---| | claude-sonnet-4-5 | 3.00 | 15.00 | main agent | | claude-haiku-4-5 | 1.00 | 5.00 | classifier, entity, rerank | | text-embedding-3-large | 0.13 | n/a | semantic embedding | | rerank-multilingual-v3.0 | n/a | n/a | $1 per 1,000 search units |

Real pricing follows published provider rates. If rates change, update the TOKEN_PRICING / UNIT_PRICING tables in apps/web/src/lib/ai-cost-tracking.ts.

Expected result

The platform owner can:

• monitor daily/monthly AI cost
• identify which model weighs most on budget (typically chat_main Sonnet)
• see which tenants generate the most AI calls
• spotlight anomalous sessions (very high cost)
• make data-driven decisions on enabling/disabling reranker and other LLM features

Common errors

• Empty dashboard with warning "ai_usage_events table not yet provisioned"
• cause: migration 0071 not applied on Supabase
• solution: apply the migration via Supabase Studio (SQL Editor) or supabase db push

• Zero cost on visible calls
• cause: the model is not in the pricing table (e.g., a new provider not mapped)
• solution: update TOKEN_PRICING or UNIT_PRICING in ai-cost-tracking.ts

• 403 Missing permission
• cause: the user lacks platform.ai.global_policy.manage
• solution: assign the permission via Platform IAM or sign in as super_admin

Out of scope (W05 known limits)

• No historical backfill: logging starts when the migration is applied
• No email alerts on cost thresholds (data-driven decision post-W06)
• No CSV export (data-driven decision)
• No end-user granularity (only per-tenant + global)
• No tenant-side dashboard (platform owner only for now)

Related guides

• roles/owner

FAQ

Can I see costs for only my tenant?

In this release the dashboard is platform-side. A tenant-side dashboard (tenant owner sees only own costs) is in the future backlog.

What does "haiku_rerank" vs "cohere_rerank" mean?

The W04 semantic reranker has two configurable providers:

• haiku_rerank uses Anthropic Haiku 4.5 (default when enabled)
• cohere_rerank uses Cohere Rerank API (requires cohere_api_key configured)

What happens if the `ai_usage_events` table doesn't exist?

Logging fail-soft: the AI call keeps working, it just isn't tracked. The dashboard renders an empty state plus a warning.

Are costs real-time?

The history is live (every call is logged within 1-2 seconds via fire-and-forget). However the dashboard runs a SELECT on every reload, so there is no streaming/websocket layer.