AI Observability — Beyond Dashboards

Most teams stop at latency and token spend. That is necessary but not sufficient.

What to instrument

• End-to-end trace_id from API gateway through retrieval, tools, and model calls.
• Structured refusal reasons and confidence bands—not just HTTP 200.
• Data freshness and schema drift alongside model drift.

Operating model

Assign an error budget to critical workflows. When burn rate spikes, roll back prompts or routing rules first—before you retrain.

This is how CloudCastNepal frames “healthy AI”: observable, bounded, and boring under stress.