First commit

specs/en/SPEC-007-Observability.md

@@ -0,0 +1,202 @@
+# SPEC-007 — Observability
+
+## Scope
+
+Observability cobre logs, Metrics, traces, Events IC/NOC/GRL, Langfuse, OpenTelemetry, dashboards, Alerts and evidências operational.
+
+## correlation
+
+fields required:
+
+```text
+request_id
+trace_id
+session_id
+conversation_key
+tenant_id
+agent_id
+channel
+message_id
+route
+intent
+```
+
+## Logs
+
+Formato:
+
+```json
+{
+  "timestamp": "2026-06-19T12:00:00Z",
+  "level": "INFO",
+  "service": "agent-runtime",
+  "event": "runtime.route.selected",
+  "tenant_id": "default",
+  "agent_id": "telecom_contas",
+  "session_id": "default:telecom_contas:session-001",
+  "trace_id": "trace-001",
+  "route": "billing_agent",
+  "intent": "billing_invoice_explanation"
+}
+```
+
+## Traces
+
+```mermaid
+flowchart TD
+    T[conversation trace] --> A[gateway.received]
+    T --> B[channel.normalized]
+    T --> C[runtime.started]
+    T --> D[guardrails.input]
+    T --> E[routing]
+    T --> F[agent.execution]
+    F --> G[mcp.tool]
+    F --> H[llm.generation]
+    T --> I[guardrails.output]
+    T --> J[judges]
+    T --> K[persist]
+```
+
+## Metrics
+
+| metric | Dimensões |
+|---|---|
+| `requests_total` | service, tenant, agent, channel, status |
+| `request_latency_ms` | service, route, intent |
+| `active_sessions` | tenant, agent |
+| `llm_tokens_total` | provider, model, profile |
+| `llm_cost_estimated` | provider, model, tenant, agent |
+| `mcp_tool_calls_total` | tool, server, status |
+| `mcp_tool_latency_ms` | tool, server |
+| `guardrail_blocks_total` | code, phase, agent |
+| `judge_scores` | metric, agent, route |
+| `errors_total` | service, component, error_type |
+
+## Langfuse
+
+Data registrados:
+
+- trace of conversation;
+- spans técnicos;
+- generations LLM;
+- prompts and responses when allowed;
+- tokens;
+- Costs;
+- Latency;
+- scores;
+- metadata;
+- erros.
+
+## OpenTelemetry
+
+Configuration:
+
+```yaml
+otel:
+  enabled: true
+  service_name: agent-runtime
+  exporter: otlp
+  endpoint: http://otel-collector:4317
+```
+
+## IC/NOC/GRL
+
+| Família | Events |
+|---|---|
+| IC | `IC.GATEWAY_RECEIVED`, `IC.AGENT_STARTED`, `IC.AGENT_COMPLETED` |
+| NOC | `NOC.RUNTIME_FAILED`, `NOC.MCP_TIMEOUT`, `NOC.LLM_FAILED` |
+| GRL | `GRL.INPUT_BLOCKED`, `GRL.OUTPUT_BLOCKED`, `GRL.MASK_APPLIED` |
+
+## Dashboards
+
+| Dashboard | Conteúdo |
+|---|---|
+| Platform Overview | traffic, erros, Latency, Sessions. |
+| Agent Runtime | rotas, intents, Memory, checkpoints. |
+| LLM Usage | tokens, Cost, Latency, provider/model. |
+| MCP Operations | Calls, erros, cache, Latency. |
+| Guardrails | Blocks, observe-only, códigos. |
+| Evals | scores, trends, regressões. |
+| Channels | traffic por canal, erros, retries. |
+
+## Alerts
+
+| Alert | Condition |
+|---|---|
+| `GatewayHighErrorRate` | Erros 5xx above of the Limit. |
+| `RuntimeLatencyHigh` | p95 above of the SLO. |
+| `LLMProviderUnavailable` | Failures consecutive of provider. |
+| `MCPToolTimeoutSpike` | increase of timeouts. |
+| `GuardrailBlockSpike` | increase abnormal of Blocks. |
+| `EvaluatorRunFailed` | run batch failed. |
+| `CheckpointFailure` | Failure persistente em checkpoint. |
+
+## Mascaramento
+
+fields mascarados:
+
+- tokens;
+- API keys;
+- senhas;
+- secrets;
+- CPF/CNPJ, when aplicável;
+- telefone, when configured;
+- payload bruto of canal;
+- documentos sensíveis.
+
+## Evidências
+
+Relatórios of homologação Include:
+
+- health checks;
+- logs of Execution;
+- traces Langfuse;
+- Metrics;
+- results of guardrails;
+- results of judges;
+- Calls MCP;
+- Calls LLM;
+- relatório of the evaluator;
+- relatório of the certification suite.
+
+
+## Requirements No Funcionais
+
+| Categoria | Requirement |
+|---|---|
+| Availability | Components deployáveis expõem `/health` and `/ready`. |
+| Escalabilidade | Apps stateless escalam horizontalmente. State conversational fica em repositórios external. |
+| Security | Segredos are fornecidos por secret store or Kubernetes Secrets. |
+| Observability | Logs, Metrics and traces usam correlation por request_id, trace_id, session_id, tenant_id and agent_id. |
+| Auditabilidade | Decisions of rota, guardrail, judge, MCP and LLM are rastreáveis. |
+| Portabilidade | Execution suportada em local, Docker Compose and Kubernetes/OKE. |
+| Configuration | Comportamento variável is controlado por `.env` and YAML versionado. |
+
+
+## Acceptance criteria
+
+- [ ] all os services emitem Structured logs.
+- [ ] Trace correlaciona gateway, runtime, MCP, LLM, guardrails and judges.
+- [ ] Langfuse recebe traces when enabled.
+- [ ] OTEL exporta spans when enabled.
+- [ ] Metrics mínimas are disponíveis.
+- [ ] Dashboards are definidos.
+- [ ] Alerts are definidos.
+- [ ] Segredos and PII are mascarados.
+- [ ] Evaluator consome Data observáveis.
+- [ ] Certification Suite gera evidências.
+
+
+## Glossário
+
+| Termo | Definição |
+|---|---|
+| Agent Platform | Plataforma composta por runtime, gateways, evaluator, templates, contracts and Components operational. |
+| Agent Framework | Biblioteca/core reutilizável with contracts, guardrails, judges, Memory, telemetria, providers and utilitários. |
+| Agent Runtime | Motor of Execution of Agents based on LangGraph, State, Session, Memory, checkpoints, Routing and ciclo of vida. |
+| Agent Gateway | Aplicação deployável of Input, Routing and orquestração between backends/Agents. |
+| Channel Gateway | Aplicação or módulo of normalização of payloads of canais for GatewayRequest. |
+| AI Gateway | Aplicação of Governance, Routing and abstração of Calls LLM/embedding. |
+| MCP Gateway | Aplicação of Governance and Routing of tools MCP. |
+| Evaluator | layer of Evaluation online/offline, regressão and Certification. |
+| Business Context | Conjunto of chaves canônicas of negócio: customer_key, contract_key, interaction_key, account_key, resource_key and session_key. |