# SPEC-007 — Observability

## Scope

Observability cobre logs, Metrics, traces, Events IC/NOC/GRL, Langfuse, OpenTelemetry, dashboards, Alerts and evidências operational.

## correlation

fields required:

```text
request_id
trace_id
session_id
conversation_key
tenant_id
agent_id
channel
message_id
route
intent
```

## Logs

Formato:

```json
{
  "timestamp": "2026-06-19T12:00:00Z",
  "level": "INFO",
  "service": "agent-runtime",
  "event": "runtime.route.selected",
  "tenant_id": "default",
  "agent_id": "telecom_contas",
  "session_id": "default:telecom_contas:session-001",
  "trace_id": "trace-001",
  "route": "billing_agent",
  "intent": "billing_invoice_explanation"
}
```

## Traces

```mermaid
flowchart TD
    T[conversation trace] --> A[gateway.received]
    T --> B[channel.normalized]
    T --> C[runtime.started]
    T --> D[guardrails.input]
    T --> E[routing]
    T --> F[agent.execution]
    F --> G[mcp.tool]
    F --> H[llm.generation]
    T --> I[guardrails.output]
    T --> J[judges]
    T --> K[persist]
```

## Metrics

| metric | Dimensões |
|---|---|
| `requests_total` | service, tenant, agent, channel, status |
| `request_latency_ms` | service, route, intent |
| `active_sessions` | tenant, agent |
| `llm_tokens_total` | provider, model, profile |
| `llm_cost_estimated` | provider, model, tenant, agent |
| `mcp_tool_calls_total` | tool, server, status |
| `mcp_tool_latency_ms` | tool, server |
| `guardrail_blocks_total` | code, phase, agent |
| `judge_scores` | metric, agent, route |
| `errors_total` | service, component, error_type |

## Langfuse

Data registrados:

- trace of conversation;
- spans técnicos;
- generations LLM;
- prompts and responses when allowed;
- tokens;
- Costs;
- Latency;
- scores;
- metadata;
- erros.

## OpenTelemetry

Configuration:

```yaml
otel:
  enabled: true
  service_name: agent-runtime
  exporter: otlp
  endpoint: http://otel-collector:4317
```

## IC/NOC/GRL

| Família | Events |
|---|---|
| IC | `IC.GATEWAY_RECEIVED`, `IC.AGENT_STARTED`, `IC.AGENT_COMPLETED` |
| NOC | `NOC.RUNTIME_FAILED`, `NOC.MCP_TIMEOUT`, `NOC.LLM_FAILED` |
| GRL | `GRL.INPUT_BLOCKED`, `GRL.OUTPUT_BLOCKED`, `GRL.MASK_APPLIED` |

## Dashboards

| Dashboard | Conteúdo |
|---|---|
| Platform Overview | traffic, erros, Latency, Sessions. |
| Agent Runtime | rotas, intents, Memory, checkpoints. |
| LLM Usage | tokens, Cost, Latency, provider/model. |
| MCP Operations | Calls, erros, cache, Latency. |
| Guardrails | Blocks, observe-only, códigos. |
| Evals | scores, trends, regressões. |
| Channels | traffic por canal, erros, retries. |

## Alerts

| Alert | Condition |
|---|---|
| `GatewayHighErrorRate` | Erros 5xx above of the Limit. |
| `RuntimeLatencyHigh` | p95 above of the SLO. |
| `LLMProviderUnavailable` | Failures consecutive of provider. |
| `MCPToolTimeoutSpike` | increase of timeouts. |
| `GuardrailBlockSpike` | increase abnormal of Blocks. |
| `EvaluatorRunFailed` | run batch failed. |
| `CheckpointFailure` | Failure persistente em checkpoint. |

## Mascaramento

fields mascarados:

- tokens;
- API keys;
- senhas;
- secrets;
- CPF/CNPJ, when aplicável;
- telefone, when configured;
- payload bruto of canal;
- documentos sensíveis.

## Evidências

Relatórios of homologação Include:

- health checks;
- logs of Execution;
- traces Langfuse;
- Metrics;
- results of guardrails;
- results of judges;
- Calls MCP;
- Calls LLM;
- relatório of the evaluator;
- relatório of the certification suite.


## Requirements No Funcionais

| Categoria | Requirement |
|---|---|
| Availability | Components deployáveis expõem `/health` and `/ready`. |
| Escalabilidade | Apps stateless escalam horizontalmente. State conversational fica em repositórios external. |
| Security | Segredos are fornecidos por secret store or Kubernetes Secrets. |
| Observability | Logs, Metrics and traces usam correlation por request_id, trace_id, session_id, tenant_id and agent_id. |
| Auditabilidade | Decisions of rota, guardrail, judge, MCP and LLM are rastreáveis. |
| Portabilidade | Execution suportada em local, Docker Compose and Kubernetes/OKE. |
| Configuration | Comportamento variável is controlado por `.env` and YAML versionado. |


## Acceptance criteria

- [ ] all os services emitem Structured logs.
- [ ] Trace correlaciona gateway, runtime, MCP, LLM, guardrails and judges.
- [ ] Langfuse recebe traces when enabled.
- [ ] OTEL exporta spans when enabled.
- [ ] Metrics mínimas are disponíveis.
- [ ] Dashboards are definidos.
- [ ] Alerts are definidos.
- [ ] Segredos and PII are mascarados.
- [ ] Evaluator consome Data observáveis.
- [ ] Certification Suite gera evidências.


## Glossário

| Termo | Definição |
|---|---|
| Agent Platform | Plataforma composta por runtime, gateways, evaluator, templates, contracts and Components operational. |
| Agent Framework | Biblioteca/core reutilizável with contracts, guardrails, judges, Memory, telemetria, providers and utilitários. |
| Agent Runtime | Motor of Execution of Agents based on LangGraph, State, Session, Memory, checkpoints, Routing and ciclo of vida. |
| Agent Gateway | Aplicação deployável of Input, Routing and orquestração between backends/Agents. |
| Channel Gateway | Aplicação or módulo of normalização of payloads of canais for GatewayRequest. |
| AI Gateway | Aplicação of Governance, Routing and abstração of Calls LLM/embedding. |
| MCP Gateway | Aplicação of Governance and Routing of tools MCP. |
| Evaluator | layer of Evaluation online/offline, regressão and Certification. |
| Business Context | Conjunto of chaves canônicas of negócio: customer_key, contract_key, interaction_key, account_key, resource_key and session_key. |