Deployment & Operations
Production rollout, cost monitoring, observability, model rotation, and the day-to-day work of keeping deployed systems within bounds.
No practices in this stage match the current filters.
-
WhenA production agent has produced a critical failure and you are conducting root cause analysis.
UseScore the context that was in place at the time of the failure against the seven criteria: role clarity, guardrail coverage, instruction consistency, tool-schema quality, grounding sufficiency, injection hardening, and token efficiency. The lowest-scoring dimension predicts the most likely failure class. Hallucination failures trace primarily to grounding sufficiency. Manipulation or injection failures trace to guardrail coverage and injection hardening. Tool misuse failures trace to tool-schema quality. Use this to narrow the root cause search before reading logs, because the context is static and available for inspection while execution logs require reconstruction.
EvidenceIn the Bousetouane (2026) controlled study, each of seven context criteria was validated as a matched predictor of its corresponding behavioral failure class across 300 multi-turn sessions and three deployment domains. The predictive relationship was established non-circularly: context scores were recorded before behavioral tests ran. The result shows context quality as a leading indicator, not merely a correlate, of specific failure types.
-
WhenYou are planning to adopt a context-compression layer (command-output compressor, retrieval ranker, or API-boundary proxy) for an API-based coding or reasoning agent and your current evaluation metric is token count removed.
UseRun a controlled experiment with actual provider billing data as the primary outcome metric before adopting the intervention. Capture the four-component cost breakdown per session (new input tokens, cached input tokens, output tokens, overhead) and compare the metered invoice across a baseline and intervention cohort on the same task set. Accept the intervention only if billed cost per successfully completed task decreases. Token count delta is a secondary metric; billed cost delta is the decision variable.
EvidenceWeinberger and Hozez (2026) ran 2,848 metered Claude Code sessions across 103 tasks, seven repositories, and three models. A command-output compressor that removed 38.4% of tool-output tokens raised billed costs by 6.8%. Prompt-cache traffic accounted for roughly 87% of the reconstructed four-component cost and roughly 80% of the actual invoice. The compression targeted a cost component that constituted a minority of spend, while feedback loops (the agent re-reading files or re-searching after losing compressed context) generated additional uncached input charges that more than offset the savings.