agent_platform_oci/specs/en/SPEC-002-Agent-Runtime.md

# SPEC-002 — Agent Runtime

## 1. Overview

The Agent Runtime coordinates conversational execution for agents on the OCI platform. It combines LangGraph graphs, per-conversation state, intent routing, domain agents, guardrails, judges, long-term memory, and persistence. The reusable framework library provides services and contracts consumed by agent backends; the reference `agent_template_backend` assembles these services into an executable workflow.

## 2. Implementation structure

| Component | Location | Responsibility |
|---|---|---|
| Shared runtime | `libs/agent_framework/src/agent_framework/runtime/` | Reusable agent-execution capabilities. |
| Reference workflow | `templates/agent_template_backend/app/workflows/agent_graph.py` | Graph construction and workflow-node implementations. |
| Workflow state | `templates/agent_template_backend/app/state.py` | Typed definition of data exchanged between nodes. |
| Agent configuration | `templates/agent_template_backend/config/agents.yaml` | Agent registry and policy/configuration file paths. |
| Routing configuration | `templates/agent_template_backend/config/routing.yaml` | Router mode, fallback, threshold, and intent catalog. |
| Tool configuration | `templates/agent_template_backend/config/tools.yaml` | Tool definitions used by agents. |

## 3. Execution state

The workflow uses the `AgentState` definition shown below. Its `TypedDict(total=False)` declaration allows nodes to add the fields relevant to each execution stage without requiring every field to be present in each state instance.

```python
from typing import Any, TypedDict

class AgentState(TypedDict, total=False):
    tenant_id: str
    agent_id: str
    session_id: str
    conversation_key: str
    workflow_id: str
    agent_profile: dict[str, Any]
    user_text: str
    sanitized_input: str
    route: str
    intent: str
    route_decision: dict[str, Any]
    answer: str
    final_answer: str
    history: list[dict[str, Any]]
    context: dict[str, Any]
    guardrail_decisions: list[dict[str, Any]]
    judge_results: list[dict[str, Any]]
    next_state: str
    domain: str
    mcp_tools: list[str]
    mcp_results: list[dict[str, Any]]
    rag_results: list[dict[str, Any]]
    available_mcp_tools: list[str]
    selected_tool_call: dict[str, Any]
    pending_tool_call: dict[str, Any]
    active_transaction: dict[str, Any]
    last_transaction: dict[str, Any]
    transaction_status: str
    transaction_pre_validation: dict[str, Any]
    transaction_evidence: list[dict[str, Any]]
    last_transaction_evidence: dict[str, Any]
    relevant_transaction_evidence: list[dict[str, Any]]
    confirmation_required: bool
    confirmation_received: bool
    tool_policy_result: dict[str, Any]
    missing_parameters: list[str]
    supervisor_plan: dict[str, Any]
    supervisor_results: list[dict[str, Any]]
    agent_responses: list[dict[str, Any]]
    pending_topics: list[dict[str, Any]]
    handled_topics: list[dict[str, Any]]
    operation_results: dict[str, dict[str, Any]]
    multi_intent_plan: dict[str, Any]
    active_agent: str
    route_bypassed: bool
    continuity_signal: dict[str, Any]
    session_control: str
    session_ended: bool
    human_handoff_requested: bool
    blocked: bool
    supervisor_action: str
    supervisor_guidance: str
    supervisor_attempt: int
    supervisor_handover_reason: str
    output_supervisor_results: list[dict[str, Any]]
    output_guardrails_already_applied: bool
    long_term_memories: list[dict[str, Any]]
    long_term_memory_context: str
    long_term_memory_write_result: dict[str, Any]
    long_term_memory_subject_key: str
    long_term_memory_load_error: str
```

## 4. Execution flow

The workflow is compiled using `FrameworkStateGraph(AgentState)` with a checkpointer created by `create_langgraph_checkpointer(self.settings)`.

```mermaid
flowchart TD
  START([START]) --> input_guardrails
  input_guardrails -->|continue| load_long_term_memory
  input_guardrails -->|blocked| output_guardrails
  load_long_term_memory --> routing_decision
  routing_decision --> billing_agent
  routing_decision --> product_agent
  routing_decision --> orders_agent
  routing_decision --> support_agent
  routing_decision --> handoff
  routing_decision --> human_handoff
  routing_decision --> end_session
  routing_decision --> supervisor_agent
  billing_agent --> output_supervisor
  product_agent --> output_supervisor
  orders_agent --> output_supervisor
  support_agent --> output_supervisor
  handoff --> output_supervisor
  human_handoff --> output_supervisor
  end_session --> output_supervisor
  supervisor_agent --> output_supervisor
  output_supervisor --> output_guardrails
  output_guardrails -->|continue| judge
  output_guardrails -->|blocked| persist
  judge --> supervisor_review
  supervisor_review --> persist_long_term_memory
  persist_long_term_memory --> persist
  persist --> END([END])
```

Routing after `input_guardrails` depends on `state['blocked']`. Following `routing_decision`, the value of `state['route']` selects one of the registered nodes, defaulting to `billing_agent` when the key is absent. Routing after `output_guardrails` also checks `state['blocked']`.

## 5. Workflow nodes

| Node | Responsibility |
|---|---|
| `input_guardrails` | Evaluates and processes input before routing. |
| `load_long_term_memory` | Loads long-term memory context. |
| `routing_decision` | Determines the conversation's execution destination. |
| `billing_agent` | Runs the billing agent. |
| `product_agent` | Runs the product and services agent. |
| `orders_agent` | Runs the orders agent. |
| `support_agent` | Runs the support agent. |
| `handoff` | Handles the transfer path defined by the workflow. |
| `human_handoff` | Handles requests for transfer to a human representative. |
| `end_session` | Handles session termination. |
| `supervisor_agent` | Runs the supervisor-related path. |
| `output_supervisor` | Processes response supervision. |
| `output_guardrails` | Applies output policies. |
| `judge` | Evaluates the response. |
| `supervisor_review` | Processes review after the judge. |
| `persist_long_term_memory` | Performs the long-term memory write stage. |
| `persist` | Performs the final workflow persistence stage. |

Nodes are wrapped by the `_node` method, which executes them within `self.langgraph_telemetry.node(name, state)`.

## 6. Routing

The `config/routing.yaml` file defines the main options under the `router` key:

```yaml
# Configurable enterprise routing with MCP-aware intents.
router:
  # mode can also be set through the ROUTING_MODE environment variable.
  # Values: router | supervisor
  mode: router
  fallback_agent: billing_agent
  confidence_threshold: 0.65
  allow_handoff: true
```

The configuration also contains `state_policies`, which keep certain confirmation or parameter-collection stages bound to the corresponding agent, and an `intents` catalog with attributes such as `name`, `domain`, `agent`, `priority`, `mcp_tools`, `keywords`, and `examples`.

The agent catalog in `config/agents.yaml` declares `telecom_contas` and `retail_orders`. Each entry points to prompt, routing, guardrail, judge, MCP server, and tool policy/configuration files, as well as domain metadata.

## 7. Agent and tool execution

Domain nodes dispatch requests to the billing, product, order, and support agent instances. MCP tools are exposed to agents through tool configuration and tool-routing components. Tool results can be carried in `mcp_results`; selected tool names are stored in `mcp_tools`.

State also supports `selected_tool_call`, `pending_tool_call`, `tool_policy_result`, `missing_parameters`, and transaction-evidence fields. These allow preparation, validation, confirmation, and execution stages to share information across the conversation.

## 8. Transaction state

`AgentState` holds the current transaction in `active_transaction` and the latest completed transaction in `last_transaction`. The fields `transaction_status`, `transaction_pre_validation`, `transaction_evidence`, `last_transaction_evidence`, `relevant_transaction_evidence`, `confirmation_required`, and `confirmation_received` carry the information required for transactional processing.

Routing configuration defines state policies for `WAITING_BILLING_CONFIRMATION`, `WAITING_PRODUCT_CONFIRMATION`, `WAITING_ORDER_CONFIRMATION`, `WAITING_SUPPORT_CONFIRMATION`, `COLLECTING_BILLING_PARAMETERS`, `COLLECTING_PRODUCT_PARAMETERS`, `COLLECTING_ORDER_PARAMETERS`, and `COLLECTING_SUPPORT_PARAMETERS`.

The transactional-flow reference guide is [`docs/TRANSACTION_STATE_DEVELOPER_GUIDE.md`](../docs/TRANSACTION_STATE_DEVELOPER_GUIDE.md).

## 9. Memory and checkpoints

The workflow includes two explicit long-term memory stages: `load_long_term_memory`, before routing, and `persist_long_term_memory`, before final persistence. The fields `long_term_memories`, `long_term_memory_context`, `long_term_memory_write_result`, `long_term_memory_subject_key`, and `long_term_memory_load_error` form this capability's state interface.

Graph construction passes the result of `create_langgraph_checkpointer(self.settings)` to `compile`. Checkpoint storage and retrieval are handled by the configured checkpointer, retaining state continuity across interactions according to the selected backend.

## 10. Guardrails, supervision, and judges

Input passes through `input_guardrails` before memory loading and routing. When input is blocked, execution goes directly to `output_guardrails`.

Agent responses pass through `output_supervisor` and `output_guardrails`. An unblocked response proceeds to `judge` and `supervisor_review`; a blocked response goes to `persist`. Results are carried in `guardrail_decisions`, `output_supervisor_results`, `judge_results`, `supervisor_action`, `supervisor_guidance`, and related state fields.

## 11. Observability

The workflow wraps each node in `langgraph_telemetry.node` instrumentation, associating execution with the node identity and conversation state. Framework telemetry and observability mechanisms provide traceability for decisions and execution stages.

## 12. Error handling and recovery by component

### 12.1. Failure propagation principles

The runtime distinguishes failures handled within a stage, negative policy/tool results, and exceptions that prevent graph execution. Each component's contract determines whether processing continues, a response is modified, or the exception is returned to the caller. The `blocked` field represents a policy decision in state; it does not generically denote a Python exception.

| Area | Representation | Execution impact |
|---|---|---|
| Guardrails | Decisions with `allowed`, `code`, and `reason`; supervision decisions with `action` | May stop routing, replace or sanitize the response, or allow processing to continue. |
| MCP | Tool results with `ok`, `error`, `tool_name`, and source-specific execution details | The agent uses the result to control how the operation proceeds. |
| Long-term memory | `long_term_memory_load_error` and `long_term_memory_write_result` fields | Read or write exceptions caught in the node do not terminate graph execution. |
| RAG | Retrieval result or technical-error object | A failure may be incorporated into the agent's execution context. |
| Checkpoints | Repository exceptions, integrity validation, and version recovery | Recovery may select an earlier valid checkpoint or start from a clean state; exhausted storage retries propagate an exception. |
| Workflow | Exceptions raised by nodes or `graph.ainvoke` | Execution is marked as failed in telemetry and the exception is re-raised. |

### 12.2. Input, sessions, and input guardrails

The `input_guardrails` node first checks `session_ended`. If the session has already ended, it sets `blocked=True`, `session_control="END_SESSION"`, `session_ended=True`, and `next_state="SESSION_ENDED"`, composes the message configured in `SESSION_ALREADY_ENDED_MESSAGE`, and records `session.message.rejected_after_end`. This prevents the message from reaching the router or domain agents.

For active sessions, the pipeline evaluates guardrails and stores decisions in `guardrail_decisions`. If any decision has `allowed=False`, the node generates a safe response, sets `route="blocked"` and `intent="input_guardrail_blocked"`, clears the turn's MCP tool and result lists, and follows the output branch. `_input_guardrail_user_message` has specific messages for `COER`, `INPUT_SIZE`, `DLEX_IN`, `PINJ`, `TOX`, and `CMP`, plus a default response for other codes. Detailed reasons remain in decisions and telemetry; the runtime controls the user-facing message.

Evaluation records `guardrails.input.completed`, individual decisions in guardrail telemetry, and GRL events through the observer. When input is blocked, the pre-routing memory node, router, and agent tools do not execute. The branch proceeds to `output_guardrails` and, according to graph conditions, ends in `persist`.

**Implementation reference:** `templates/agent_template_backend/app/workflows/agent_graph.py` (`input_guardrails`, `_input_guardrail_user_message`, `_after_input_guardrails`).

### 12.3. Routing, classification, and continuity

Routing configuration is loaded through `routing/config_loader.py`; a missing file raises `FileNotFoundError`. `EnterpriseRouter` locally handles errors from classification calls and model-output parsing. In the semantic classifier for expected inputs, a failed `llm.ainvoke` call is logged with `logger.warning`, and the function returns `(None, None)`. Subsequent routing paths can therefore operate without treating the failure as a valid classification.

The flow uses `fallback_agent` and `confidence_threshold` settings, as well as state policies in `routing.yaml`. Selecting a fallback is a routing decision, not intrinsically an error code. The `route_decision`, `continuity_signal`, `pending_topics`, and `multi_intent_plan` structures preserve information for resumption and routing of related turns.

Failure recovery depends on the method actually invoked: not every internal routing error is handled by the same `except` clause. Exceptions escaping the node reach the workflow failure handler in `ainvoke`.

**Implementation references:** `libs/agent_framework/src/agent_framework/routing/config_loader.py`, `routing/enterprise_router.py`, `routing/continuity.py`; `templates/agent_template_backend/app/workflows/agent_graph.py` (`routing_decision`, `ainvoke`).

### 12.4. Agent execution, arguments, and MCP tools

Tool preparation is handled by the reusable runtime in `runtime/agent_runtime.py`. `_prepare_mcp_call` invokes the MCP router and returns a structured preparation error when `prepare_call` provides one. If the tool router is unavailable, the runtime produces a negative result with `ok=False`, `tool_name`, and the description `MCP Tool Router indisponível`.

During parameter mapping, regular-expression errors (`re.error`, `IndexError`, `ValueError`) are caught and logged as `mcp.parameter.regex_extract_failed`. For LLM-based extraction, missing model/message data and inference exceptions are recorded as `mcp.parameter.llm_extract_failed`. Argument processing and call results remain subject to tool validation and agent policies; absence of an exception alone does not establish success.

MCP results may expose `ok=False` and `error`, including validation and execution failures. The implementation emits `NOC.MCP_TOOL_FAILED` along one path that aggregates negative tool results. Transaction handling, confirmations, evidence, and partial results operate on execution data including `active_transaction`, `transaction_pre_validation`, `transaction_evidence`, and `operation_results`, depending on operation type.

**Implementation reference:** `libs/agent_framework/src/agent_framework/runtime/agent_runtime.py` (`_prepare_mcp_call`, parameter handling, tool execution, and result aggregation).

### 12.5. RAG and knowledge retrieval

The runtime wraps knowledge-retrieval failures in a technical result containing `technical_error=True`, `technical_error_in_rag=True`, `status="error"`, and a description in `error`. This allows the agent to distinguish retrieval failure from a completed search yielding no useful evidence.

The RAG service also validates preconditions: `RagService.generate_answer` raises `RuntimeError` when used without an LLM, and `kbdb_service.py` checks configuration/signatures and disallows ingestion/publication when the KBDB provider is configured exclusively for serving. During ingestion, a missing directory may raise `FileNotFoundError`, while `chunk_overlap >= chunk_size` raises `ValueError`. These initialization and administration errors must not be confused with an empty retrieval result during a conversation.

**Implementation references:** `libs/agent_framework/src/agent_framework/runtime/agent_runtime.py` (technical RAG error handling); `rag/rag_service.py`, `rag/kbdb_service.py`, `rag/ingest.py`.

### 12.6. Long-term memory

The `load_long_term_memory` node catches exceptions from `long_term_memory_manager.load(state)`. If loading fails, it publishes `long_term_memory.load.failed`, returns `long_term_memories=[]` and `long_term_memory_context=""`, and stores the exception text in `long_term_memory_load_error`. The conversation continues without retrieved long-term memory context for that turn.

The `persist_long_term_memory` node catches exceptions from `long_term_memory_manager.persist_turn(state)`. A successful write publishes `long_term_memory.persist.completed` and stores its return value in `long_term_memory_write_result`; a failure publishes `long_term_memory.persist.failed` and returns `{"saved": 0, "error": str(exc)}` in the same field. Thus, a long-term memory persistence failure is isolated within the node and does not automatically terminate the graph with an exception.

These fields concern the workflow's long-term memory and are separate from LangGraph checkpoint persistence.

**Implementation reference:** `templates/agent_template_backend/app/workflows/agent_graph.py` (`load_long_term_memory`, `persist_long_term_memory`).

### 12.7. Checkpoints, retries, and integrity

The graph is compiled with the checkpointer created by `create_langgraph_checkpointer(self.settings)` and uses `conversation_key` or `session_id` as `thread_id` in `ainvoke`. The repository layer in `checkpoints/checkpoint_repository.py` retries operations protected by `_with_retry`: after an exception, it computes a bounded exponential delay with jitter, records `checkpoint.<operation>.retry`, and retries up to `retry_policy.max_attempts`. Once attempts are exhausted, it re-raises the last exception.

On recovery, `recover_latest` scans recent records, discards those failing `is_valid_checkpoint`, and skips checkpoints with `CheckpointIntegrityError`, recording `checkpoint.recovery.skip_corrupt`. If no valid checkpoint remains after integrity failures, recovery returns `None` so execution can begin without restored state. Compaction after a write is isolated: a failure records `checkpoint.compaction.failed` and does not undo the completed write.

Integrity recovery cannot guarantee retention of history if no valid checkpoint can be recovered. Storage-communication errors that exceed the retry policy follow the exception path of the calling component.

**Implementation references:** `libs/agent_framework/src/agent_framework/checkpoints/checkpoint_repository.py`, `checkpoints/langgraph_saver.py`; `templates/agent_template_backend/app/workflows/agent_graph.py` (`ainvoke`).

### 12.8. Output supervision and output guardrails

`output_supervisor` evaluates the response and interprets `RailAction`. For `ALLOW`, `SANITIZE`, and `OBSERVE`, it uses `decision.candidate`; for `HANDOVER`, it supplies a message referring the user to a specialist; for other actions, it uses `decision.fallback_message`. The action, guidance, and results are stored in `supervisor_action`, `supervisor_guidance`, and `output_supervisor_results`.

The node sets `output_guardrails_already_applied=True`. When `output_guardrails` sees this flag, it returns the prepared response without running the pipeline again. Otherwise, it runs `guardrails.run_output`, records decisions, blocks, and GRL completion, then returns `final_answer`.

The parallel rail executor (`guardrails/parallel_executor.py`) converts unexpected task failures to `RailResult`: it selects `RailAction.BLOCK` when `fail_closed` is enabled and `RailAction.OBSERVE` otherwise. This is executor policy, not a claim that all guardrails fail closed. The executor also handles cancellation and early termination when a terminal action is produced.

**Implementation references:** `templates/agent_template_backend/app/workflows/agent_graph.py` (`output_supervisor`, `output_guardrails`); `libs/agent_framework/src/agent_framework/guardrails/parallel_executor.py`.

### 12.9. Judges and supervisor review

The `judge` node builds evaluation context from MCP results, RAG results, and relevant transaction evidence, executes evaluation, and stores outcomes in `judge_results`. The subsequent review invokes `self.supervisor.review` and records `supervisor.review.completed`. A judge's failing score or recommendation belongs to evaluation and review; it is not, by definition, an infrastructure exception.

The judge layer validates configuration and structured responses. The calibrated judge client raises `ValueError` for unsupported tasks or invalid structured output. LLM construction and loading of judge configuration can also raise `RuntimeError` or `ValueError` under the conditions checked in `judges/judge.py`. Exceptions not caught within the node propagate to `ainvoke`.

**Implementation references:** `templates/agent_template_backend/app/workflows/agent_graph.py` (`judge`, `supervisor_review`); `libs/agent_framework/src/agent_framework/judges/judge.py`, `judges/calibrated/llm_client.py`.

### 12.10. Final persistence and workflow-level failure

The `persist` node emits completion observability, including the IC event `AGENT_COMPLETED`. The public graph entry point is `ainvoke`, which configures the thread, starts workflow telemetry, and emits the start events `NOC.001` and `AGENT_STARTED` before calling `self.graph.ainvoke`.

If `graph.ainvoke` completes, the workflow is marked completed and its final state is returned. If an exception escapes the graph, the `except Exception` handler in `ainvoke` calls `workflow_telemetry.failed("agent_workflow", exc)`, emits `NOC.005` with `error` and `exception_type`, and re-raises the same exception. This path does not automatically convert the error into a successful conversational response.

Failures handled inside nodes, such as long-term memory failures, do not trigger the global `except`. Unhandled errors in routing, agent execution, checkpoint storage, or other stages may reach this boundary and cause a workflow-level failure.

**Implementation reference:** `templates/agent_template_backend/app/workflows/agent_graph.py` (`persist`, `ainvoke`).

### 12.11. Operational events and tracing

`_node` wraps every node execution in `self.langgraph_telemetry.node(name, state)`. Additional mechanisms use workflow spans, semantic events, and the observer for GRL, IC, and NOC. Concrete records in the flow include `session.message.rejected_after_end`, `guardrails.input.completed`, `guardrails.output.completed`, `output_supervisor.completed`, `supervisor.review.completed`, `long_term_memory.load.failed`, `long_term_memory.persist.failed`, `long_term_memory.persist.completed`, and `workflow_telemetry.failed` on workflow-level failure.

Operational errors should be correlated using `conversation_key` or `session_id`, together with `tenant_id` and `agent_id` when available. A memory or tool failure event does not necessarily indicate overall workflow failure; this depends on whether the component returned a state/result or propagated an exception.

**Implementation references:** `templates/agent_template_backend/app/workflows/agent_graph.py`; `libs/agent_framework/src/agent_framework/observability/`.

## 13. Integration interfaces

| Interface | Runtime integration |
|---|---|
| LangGraph | State-graph construction, compilation, and execution. |
| Domain agents | Specialized billing, product, order, and support nodes. |
| MCP | Tool selection, parameter mapping, and execution by agents. |
| RAG | Knowledge retrieval incorporated into agent execution. |
| Guardrails | Input and output evaluation. |
| Judges | Evaluation after response generation. |
| Memory | Long-term information loading and writing. |
| Checkpointer | Graph-state persistence. |
| Observability | Per-node telemetry and execution tracing. |

## 14. Implementation references

- `libs/agent_framework/src/agent_framework/runtime/`
- `templates/agent_template_backend/app/state.py`
- `templates/agent_template_backend/app/workflows/agent_graph.py`
- `templates/agent_template_backend/config/agents.yaml`
- `templates/agent_template_backend/config/routing.yaml`
- `templates/agent_template_backend/config/tools.yaml`
- [`SPEC-004 — MCP Gateway`](SPEC-004-MCP-Gateway.md)
- [`SPEC-005 — Guardrails`](SPEC-005-Guardrails.md)
- [`SPEC-007 — Observability`](SPEC-007-Observability.md)
- [`SPEC-012 — Canonical Contracts`](SPEC-012-Canonical-Contracts.md)