1.5 KiB
1.5 KiB
Observability severity patch
- Central
severity_policy.pyclassifies explicit MCP failures, guardrail blocks, judge rejections, RAG failures and memory/cache errors. Telemetry.event()acceptslevelandstatus_messageand preserves WARNING/ERROR events as individual observations in compact mode.Telemetry.span()handle supportsset_status(level, message); failed MCP results mark their span ERROR without raising exceptions.- Long-term-memory read/write failures emit telemetry while preserving existing fallback behavior.
- Guardrail blocked and negative judge evaluation emit WARNING when these helper methods are invoked.
- Normal cache misses and empty RAG results are not automatically failures.
Scope / limitations
- Existing call sites must invoke
GuardrailTelemetry.blockedfor guardrail blocking events; this patch does not rewrite every pipeline call site. - Redis cache currently has no injected Telemetry instance, so its existing logger errors are not yet Langfuse observations. A separate dependency-injection change is required to cover that path safely.
- RAG database connection errors that are caught by external KBDB integrations without emitting a telemetry event cannot be classified automatically.
- No live Langfuse ingestion test was possible in this environment. Verify both compact and verbose modes against your server.
Validation
PYTHONPATH=libs/agent_framework/src python -m pytest -q tests/test_observability_severity_policy.py