agent_platform_oci/specs/en/SPEC-007-Observability.md

SPEC-007 — Observability

Scope

Observability cobre logs, Metrics, traces, Events IC/NOC/GRL, Langfuse, OpenTelemetry, dashboards, Alerts and evidências operational.

correlation

fields required:

request_id
trace_id
session_id
conversation_key
tenant_id
agent_id
channel
message_id
route
intent

Logs

Formato:

{
  "timestamp": "2026-06-19T12:00:00Z",
  "level": "INFO",
  "service": "agent-runtime",
  "event": "runtime.route.selected",
  "tenant_id": "default",
  "agent_id": "telecom_contas",
  "session_id": "default:telecom_contas:session-001",
  "trace_id": "trace-001",
  "route": "billing_agent",
  "intent": "billing_invoice_explanation"
}

Traces

flowchart TD
    T[conversation trace] --> A[gateway.received]
    T --> B[channel.normalized]
    T --> C[runtime.started]
    T --> D[guardrails.input]
    T --> E[routing]
    T --> F[agent.execution]
    F --> G[mcp.tool]
    F --> H[llm.generation]
    T --> I[guardrails.output]
    T --> J[judges]
    T --> K[persist]

Metrics

metric	Dimensões
requests_total	service, tenant, agent, channel, status
request_latency_ms	service, route, intent
active_sessions	tenant, agent
llm_tokens_total	provider, model, profile
llm_cost_estimated	provider, model, tenant, agent
mcp_tool_calls_total	tool, server, status
mcp_tool_latency_ms	tool, server
guardrail_blocks_total	code, phase, agent
judge_scores	metric, agent, route
errors_total	service, component, error_type

Langfuse

Data registrados:

• trace of conversation;
• spans técnicos;
• generations LLM;
• prompts and responses when allowed;
• tokens;
• Costs;
• Latency;
• scores;
• metadata;
• erros.

OpenTelemetry

Configuration:

otel:
  enabled: true
  service_name: agent-runtime
  exporter: otlp
  endpoint: http://otel-collector:4317

IC/NOC/GRL

Família	Events
IC	IC.GATEWAY_RECEIVED, IC.AGENT_STARTED, IC.AGENT_COMPLETED
NOC	NOC.RUNTIME_FAILED, NOC.MCP_TIMEOUT, NOC.LLM_FAILED
GRL	GRL.INPUT_BLOCKED, GRL.OUTPUT_BLOCKED, GRL.MASK_APPLIED

Dashboards

Dashboard	Conteúdo
Platform Overview	traffic, erros, Latency, Sessions.
Agent Runtime	rotas, intents, Memory, checkpoints.
LLM Usage	tokens, Cost, Latency, provider/model.
MCP Operations	Calls, erros, cache, Latency.
Guardrails	Blocks, observe-only, códigos.
Evals	scores, trends, regressões.
Channels	traffic por canal, erros, retries.

Alerts

Alert	Condition
GatewayHighErrorRate	Erros 5xx above of the Limit.
RuntimeLatencyHigh	p95 above of the SLO.
LLMProviderUnavailable	Failures consecutive of provider.
MCPToolTimeoutSpike	increase of timeouts.
GuardrailBlockSpike	increase abnormal of Blocks.
EvaluatorRunFailed	run batch failed.
CheckpointFailure	Failure persistente em checkpoint.

Mascaramento

fields mascarados:

• tokens;
• API keys;
• senhas;
• secrets;
• CPF/CNPJ, when aplicável;
• telefone, when configured;
• payload bruto of canal;
• documentos sensíveis.

Evidências

Relatórios of homologação Include:

• health checks;
• logs of Execution;
• traces Langfuse;
• Metrics;
• results of guardrails;
• results of judges;
• Calls MCP;
• Calls LLM;
• relatório of the evaluator;
• relatório of the certification suite.

Requirements No Funcionais

Categoria	Requirement
Availability	Components deployáveis expõem /health and /ready.
Escalabilidade	Apps stateless escalam horizontalmente. State conversational fica em repositórios external.
Security	Segredos are fornecidos por secret store or Kubernetes Secrets.
Observability	Logs, Metrics and traces usam correlation por request_id, trace_id, session_id, tenant_id and agent_id.
Auditabilidade	Decisions of rota, guardrail, judge, MCP and LLM are rastreáveis.
Portabilidade	Execution suportada em local, Docker Compose and Kubernetes/OKE.
Configuration	Comportamento variável is controlado por .env and YAML versionado.

Acceptance criteria

• all os services emitem Structured logs.
• Trace correlaciona gateway, runtime, MCP, LLM, guardrails and judges.
• Langfuse recebe traces when enabled.
• OTEL exporta spans when enabled.
• Metrics mínimas are disponíveis.
• Dashboards are definidos.
• Alerts are definidos.
• Segredos and PII are mascarados.
• Evaluator consome Data observáveis.
• Certification Suite gera evidências.

Glossário

Termo	Definição
Agent Platform	Plataforma composta por runtime, gateways, evaluator, templates, contracts and Components operational.
Agent Framework	Biblioteca/core reutilizável with contracts, guardrails, judges, Memory, telemetria, providers and utilitários.
Agent Runtime	Motor of Execution of Agents based on LangGraph, State, Session, Memory, checkpoints, Routing and ciclo of vida.
Agent Gateway	Aplicação deployável of Input, Routing and orquestração between backends/Agents.
Channel Gateway	Aplicação or módulo of normalização of payloads of canais for GatewayRequest.
AI Gateway	Aplicação of Governance, Routing and abstração of Calls LLM/embedding.
MCP Gateway	Aplicação of Governance and Routing of tools MCP.
Evaluator	layer of Evaluation online/offline, regressão and Certification.
Business Context	Conjunto of chaves canônicas of negócio: customer_key, contract_key, interaction_key, account_key, resource_key and session_key.