25 KiB
SPEC-002 — Agent Runtime
1. Overview
The Agent Runtime coordinates conversational execution for agents on the OCI platform. It combines LangGraph graphs, per-conversation state, intent routing, domain agents, guardrails, judges, long-term memory, and persistence. The reusable framework library provides services and contracts consumed by agent backends; the reference agent_template_backend assembles these services into an executable workflow.
2. Implementation structure
| Component | Location | Responsibility |
|---|---|---|
| Shared runtime | libs/agent_framework/src/agent_framework/runtime/ |
Reusable agent-execution capabilities. |
| Reference workflow | templates/agent_template_backend/app/workflows/agent_graph.py |
Graph construction and workflow-node implementations. |
| Workflow state | templates/agent_template_backend/app/state.py |
Typed definition of data exchanged between nodes. |
| Agent configuration | templates/agent_template_backend/config/agents.yaml |
Agent registry and policy/configuration file paths. |
| Routing configuration | templates/agent_template_backend/config/routing.yaml |
Router mode, fallback, threshold, and intent catalog. |
| Tool configuration | templates/agent_template_backend/config/tools.yaml |
Tool definitions used by agents. |
3. Execution state
The workflow uses the AgentState definition shown below. Its TypedDict(total=False) declaration allows nodes to add the fields relevant to each execution stage without requiring every field to be present in each state instance.
from typing import Any, TypedDict
class AgentState(TypedDict, total=False):
tenant_id: str
agent_id: str
session_id: str
conversation_key: str
workflow_id: str
agent_profile: dict[str, Any]
user_text: str
sanitized_input: str
route: str
intent: str
route_decision: dict[str, Any]
answer: str
final_answer: str
history: list[dict[str, Any]]
context: dict[str, Any]
guardrail_decisions: list[dict[str, Any]]
judge_results: list[dict[str, Any]]
next_state: str
domain: str
mcp_tools: list[str]
mcp_results: list[dict[str, Any]]
rag_results: list[dict[str, Any]]
available_mcp_tools: list[str]
selected_tool_call: dict[str, Any]
pending_tool_call: dict[str, Any]
active_transaction: dict[str, Any]
last_transaction: dict[str, Any]
transaction_status: str
transaction_pre_validation: dict[str, Any]
transaction_evidence: list[dict[str, Any]]
last_transaction_evidence: dict[str, Any]
relevant_transaction_evidence: list[dict[str, Any]]
confirmation_required: bool
confirmation_received: bool
tool_policy_result: dict[str, Any]
missing_parameters: list[str]
supervisor_plan: dict[str, Any]
supervisor_results: list[dict[str, Any]]
agent_responses: list[dict[str, Any]]
pending_topics: list[dict[str, Any]]
handled_topics: list[dict[str, Any]]
operation_results: dict[str, dict[str, Any]]
multi_intent_plan: dict[str, Any]
active_agent: str
route_bypassed: bool
continuity_signal: dict[str, Any]
session_control: str
session_ended: bool
human_handoff_requested: bool
blocked: bool
supervisor_action: str
supervisor_guidance: str
supervisor_attempt: int
supervisor_handover_reason: str
output_supervisor_results: list[dict[str, Any]]
output_guardrails_already_applied: bool
long_term_memories: list[dict[str, Any]]
long_term_memory_context: str
long_term_memory_write_result: dict[str, Any]
long_term_memory_subject_key: str
long_term_memory_load_error: str
4. Execution flow
The workflow is compiled using FrameworkStateGraph(AgentState) with a checkpointer created by create_langgraph_checkpointer(self.settings).
flowchart TD
START([START]) --> input_guardrails
input_guardrails -->|continue| load_long_term_memory
input_guardrails -->|blocked| output_guardrails
load_long_term_memory --> routing_decision
routing_decision --> billing_agent
routing_decision --> product_agent
routing_decision --> orders_agent
routing_decision --> support_agent
routing_decision --> handoff
routing_decision --> human_handoff
routing_decision --> end_session
routing_decision --> supervisor_agent
billing_agent --> output_supervisor
product_agent --> output_supervisor
orders_agent --> output_supervisor
support_agent --> output_supervisor
handoff --> output_supervisor
human_handoff --> output_supervisor
end_session --> output_supervisor
supervisor_agent --> output_supervisor
output_supervisor --> output_guardrails
output_guardrails -->|continue| judge
output_guardrails -->|blocked| persist
judge --> supervisor_review
supervisor_review --> persist_long_term_memory
persist_long_term_memory --> persist
persist --> END([END])
Routing after input_guardrails depends on state['blocked']. Following routing_decision, the value of state['route'] selects one of the registered nodes, defaulting to billing_agent when the key is absent. Routing after output_guardrails also checks state['blocked'].
5. Workflow nodes
| Node | Responsibility |
|---|---|
input_guardrails |
Evaluates and processes input before routing. |
load_long_term_memory |
Loads long-term memory context. |
routing_decision |
Determines the conversation's execution destination. |
billing_agent |
Runs the billing agent. |
product_agent |
Runs the product and services agent. |
orders_agent |
Runs the orders agent. |
support_agent |
Runs the support agent. |
handoff |
Handles the transfer path defined by the workflow. |
human_handoff |
Handles requests for transfer to a human representative. |
end_session |
Handles session termination. |
supervisor_agent |
Runs the supervisor-related path. |
output_supervisor |
Processes response supervision. |
output_guardrails |
Applies output policies. |
judge |
Evaluates the response. |
supervisor_review |
Processes review after the judge. |
persist_long_term_memory |
Performs the long-term memory write stage. |
persist |
Performs the final workflow persistence stage. |
Nodes are wrapped by the _node method, which executes them within self.langgraph_telemetry.node(name, state).
6. Routing
The config/routing.yaml file defines the main options under the router key:
# Configurable enterprise routing with MCP-aware intents.
router:
# mode can also be set through the ROUTING_MODE environment variable.
# Values: router | supervisor
mode: router
fallback_agent: billing_agent
confidence_threshold: 0.65
allow_handoff: true
The configuration also contains state_policies, which keep certain confirmation or parameter-collection stages bound to the corresponding agent, and an intents catalog with attributes such as name, domain, agent, priority, mcp_tools, keywords, and examples.
The agent catalog in config/agents.yaml declares telecom_contas and retail_orders. Each entry points to prompt, routing, guardrail, judge, MCP server, and tool policy/configuration files, as well as domain metadata.
7. Agent and tool execution
Domain nodes dispatch requests to the billing, product, order, and support agent instances. MCP tools are exposed to agents through tool configuration and tool-routing components. Tool results can be carried in mcp_results; selected tool names are stored in mcp_tools.
State also supports selected_tool_call, pending_tool_call, tool_policy_result, missing_parameters, and transaction-evidence fields. These allow preparation, validation, confirmation, and execution stages to share information across the conversation.
8. Transaction state
AgentState holds the current transaction in active_transaction and the latest completed transaction in last_transaction. The fields transaction_status, transaction_pre_validation, transaction_evidence, last_transaction_evidence, relevant_transaction_evidence, confirmation_required, and confirmation_received carry the information required for transactional processing.
Routing configuration defines state policies for WAITING_BILLING_CONFIRMATION, WAITING_PRODUCT_CONFIRMATION, WAITING_ORDER_CONFIRMATION, WAITING_SUPPORT_CONFIRMATION, COLLECTING_BILLING_PARAMETERS, COLLECTING_PRODUCT_PARAMETERS, COLLECTING_ORDER_PARAMETERS, and COLLECTING_SUPPORT_PARAMETERS.
The transactional-flow reference guide is docs/TRANSACTION_STATE_DEVELOPER_GUIDE.md.
9. Memory and checkpoints
The workflow includes two explicit long-term memory stages: load_long_term_memory, before routing, and persist_long_term_memory, before final persistence. The fields long_term_memories, long_term_memory_context, long_term_memory_write_result, long_term_memory_subject_key, and long_term_memory_load_error form this capability's state interface.
Graph construction passes the result of create_langgraph_checkpointer(self.settings) to compile. Checkpoint storage and retrieval are handled by the configured checkpointer, retaining state continuity across interactions according to the selected backend.
10. Guardrails, supervision, and judges
Input passes through input_guardrails before memory loading and routing. When input is blocked, execution goes directly to output_guardrails.
Agent responses pass through output_supervisor and output_guardrails. An unblocked response proceeds to judge and supervisor_review; a blocked response goes to persist. Results are carried in guardrail_decisions, output_supervisor_results, judge_results, supervisor_action, supervisor_guidance, and related state fields.
11. Observability
The workflow wraps each node in langgraph_telemetry.node instrumentation, associating execution with the node identity and conversation state. Framework telemetry and observability mechanisms provide traceability for decisions and execution stages.
12. Error handling and recovery by component
12.1. Failure propagation principles
The runtime distinguishes failures handled within a stage, negative policy/tool results, and exceptions that prevent graph execution. Each component's contract determines whether processing continues, a response is modified, or the exception is returned to the caller. The blocked field represents a policy decision in state; it does not generically denote a Python exception.
| Area | Representation | Execution impact |
|---|---|---|
| Guardrails | Decisions with allowed, code, and reason; supervision decisions with action |
May stop routing, replace or sanitize the response, or allow processing to continue. |
| MCP | Tool results with ok, error, tool_name, and source-specific execution details |
The agent uses the result to control how the operation proceeds. |
| Long-term memory | long_term_memory_load_error and long_term_memory_write_result fields |
Read or write exceptions caught in the node do not terminate graph execution. |
| RAG | Retrieval result or technical-error object | A failure may be incorporated into the agent's execution context. |
| Checkpoints | Repository exceptions, integrity validation, and version recovery | Recovery may select an earlier valid checkpoint or start from a clean state; exhausted storage retries propagate an exception. |
| Workflow | Exceptions raised by nodes or graph.ainvoke |
Execution is marked as failed in telemetry and the exception is re-raised. |
12.2. Input, sessions, and input guardrails
The input_guardrails node first checks session_ended. If the session has already ended, it sets blocked=True, session_control="END_SESSION", session_ended=True, and next_state="SESSION_ENDED", composes the message configured in SESSION_ALREADY_ENDED_MESSAGE, and records session.message.rejected_after_end. This prevents the message from reaching the router or domain agents.
For active sessions, the pipeline evaluates guardrails and stores decisions in guardrail_decisions. If any decision has allowed=False, the node generates a safe response, sets route="blocked" and intent="input_guardrail_blocked", clears the turn's MCP tool and result lists, and follows the output branch. _input_guardrail_user_message has specific messages for COER, INPUT_SIZE, DLEX_IN, PINJ, TOX, and CMP, plus a default response for other codes. Detailed reasons remain in decisions and telemetry; the runtime controls the user-facing message.
Evaluation records guardrails.input.completed, individual decisions in guardrail telemetry, and GRL events through the observer. When input is blocked, the pre-routing memory node, router, and agent tools do not execute. The branch proceeds to output_guardrails and, according to graph conditions, ends in persist.
Implementation reference: templates/agent_template_backend/app/workflows/agent_graph.py (input_guardrails, _input_guardrail_user_message, _after_input_guardrails).
12.3. Routing, classification, and continuity
Routing configuration is loaded through routing/config_loader.py; a missing file raises FileNotFoundError. EnterpriseRouter locally handles errors from classification calls and model-output parsing. In the semantic classifier for expected inputs, a failed llm.ainvoke call is logged with logger.warning, and the function returns (None, None). Subsequent routing paths can therefore operate without treating the failure as a valid classification.
The flow uses fallback_agent and confidence_threshold settings, as well as state policies in routing.yaml. Selecting a fallback is a routing decision, not intrinsically an error code. The route_decision, continuity_signal, pending_topics, and multi_intent_plan structures preserve information for resumption and routing of related turns.
Failure recovery depends on the method actually invoked: not every internal routing error is handled by the same except clause. Exceptions escaping the node reach the workflow failure handler in ainvoke.
Implementation references: libs/agent_framework/src/agent_framework/routing/config_loader.py, routing/enterprise_router.py, routing/continuity.py; templates/agent_template_backend/app/workflows/agent_graph.py (routing_decision, ainvoke).
12.4. Agent execution, arguments, and MCP tools
Tool preparation is handled by the reusable runtime in runtime/agent_runtime.py. _prepare_mcp_call invokes the MCP router and returns a structured preparation error when prepare_call provides one. If the tool router is unavailable, the runtime produces a negative result with ok=False, tool_name, and the description MCP Tool Router indisponível.
During parameter mapping, regular-expression errors (re.error, IndexError, ValueError) are caught and logged as mcp.parameter.regex_extract_failed. For LLM-based extraction, missing model/message data and inference exceptions are recorded as mcp.parameter.llm_extract_failed. Argument processing and call results remain subject to tool validation and agent policies; absence of an exception alone does not establish success.
MCP results may expose ok=False and error, including validation and execution failures. The implementation emits NOC.MCP_TOOL_FAILED along one path that aggregates negative tool results. Transaction handling, confirmations, evidence, and partial results operate on execution data including active_transaction, transaction_pre_validation, transaction_evidence, and operation_results, depending on operation type.
Implementation reference: libs/agent_framework/src/agent_framework/runtime/agent_runtime.py (_prepare_mcp_call, parameter handling, tool execution, and result aggregation).
12.5. RAG and knowledge retrieval
The runtime wraps knowledge-retrieval failures in a technical result containing technical_error=True, technical_error_in_rag=True, status="error", and a description in error. This allows the agent to distinguish retrieval failure from a completed search yielding no useful evidence.
The RAG service also validates preconditions: RagService.generate_answer raises RuntimeError when used without an LLM, and kbdb_service.py checks configuration/signatures and disallows ingestion/publication when the KBDB provider is configured exclusively for serving. During ingestion, a missing directory may raise FileNotFoundError, while chunk_overlap >= chunk_size raises ValueError. These initialization and administration errors must not be confused with an empty retrieval result during a conversation.
Implementation references: libs/agent_framework/src/agent_framework/runtime/agent_runtime.py (technical RAG error handling); rag/rag_service.py, rag/kbdb_service.py, rag/ingest.py.
12.6. Long-term memory
The load_long_term_memory node catches exceptions from long_term_memory_manager.load(state). If loading fails, it publishes long_term_memory.load.failed, returns long_term_memories=[] and long_term_memory_context="", and stores the exception text in long_term_memory_load_error. The conversation continues without retrieved long-term memory context for that turn.
The persist_long_term_memory node catches exceptions from long_term_memory_manager.persist_turn(state). A successful write publishes long_term_memory.persist.completed and stores its return value in long_term_memory_write_result; a failure publishes long_term_memory.persist.failed and returns {"saved": 0, "error": str(exc)} in the same field. Thus, a long-term memory persistence failure is isolated within the node and does not automatically terminate the graph with an exception.
These fields concern the workflow's long-term memory and are separate from LangGraph checkpoint persistence.
Implementation reference: templates/agent_template_backend/app/workflows/agent_graph.py (load_long_term_memory, persist_long_term_memory).
12.7. Checkpoints, retries, and integrity
The graph is compiled with the checkpointer created by create_langgraph_checkpointer(self.settings) and uses conversation_key or session_id as thread_id in ainvoke. The repository layer in checkpoints/checkpoint_repository.py retries operations protected by _with_retry: after an exception, it computes a bounded exponential delay with jitter, records checkpoint.<operation>.retry, and retries up to retry_policy.max_attempts. Once attempts are exhausted, it re-raises the last exception.
On recovery, recover_latest scans recent records, discards those failing is_valid_checkpoint, and skips checkpoints with CheckpointIntegrityError, recording checkpoint.recovery.skip_corrupt. If no valid checkpoint remains after integrity failures, recovery returns None so execution can begin without restored state. Compaction after a write is isolated: a failure records checkpoint.compaction.failed and does not undo the completed write.
Integrity recovery cannot guarantee retention of history if no valid checkpoint can be recovered. Storage-communication errors that exceed the retry policy follow the exception path of the calling component.
Implementation references: libs/agent_framework/src/agent_framework/checkpoints/checkpoint_repository.py, checkpoints/langgraph_saver.py; templates/agent_template_backend/app/workflows/agent_graph.py (ainvoke).
12.8. Output supervision and output guardrails
output_supervisor evaluates the response and interprets RailAction. For ALLOW, SANITIZE, and OBSERVE, it uses decision.candidate; for HANDOVER, it supplies a message referring the user to a specialist; for other actions, it uses decision.fallback_message. The action, guidance, and results are stored in supervisor_action, supervisor_guidance, and output_supervisor_results.
The node sets output_guardrails_already_applied=True. When output_guardrails sees this flag, it returns the prepared response without running the pipeline again. Otherwise, it runs guardrails.run_output, records decisions, blocks, and GRL completion, then returns final_answer.
The parallel rail executor (guardrails/parallel_executor.py) converts unexpected task failures to RailResult: it selects RailAction.BLOCK when fail_closed is enabled and RailAction.OBSERVE otherwise. This is executor policy, not a claim that all guardrails fail closed. The executor also handles cancellation and early termination when a terminal action is produced.
Implementation references: templates/agent_template_backend/app/workflows/agent_graph.py (output_supervisor, output_guardrails); libs/agent_framework/src/agent_framework/guardrails/parallel_executor.py.
12.9. Judges and supervisor review
The judge node builds evaluation context from MCP results, RAG results, and relevant transaction evidence, executes evaluation, and stores outcomes in judge_results. The subsequent review invokes self.supervisor.review and records supervisor.review.completed. A judge's failing score or recommendation belongs to evaluation and review; it is not, by definition, an infrastructure exception.
The judge layer validates configuration and structured responses. The calibrated judge client raises ValueError for unsupported tasks or invalid structured output. LLM construction and loading of judge configuration can also raise RuntimeError or ValueError under the conditions checked in judges/judge.py. Exceptions not caught within the node propagate to ainvoke.
Implementation references: templates/agent_template_backend/app/workflows/agent_graph.py (judge, supervisor_review); libs/agent_framework/src/agent_framework/judges/judge.py, judges/calibrated/llm_client.py.
12.10. Final persistence and workflow-level failure
The persist node emits completion observability, including the IC event AGENT_COMPLETED. The public graph entry point is ainvoke, which configures the thread, starts workflow telemetry, and emits the start events NOC.001 and AGENT_STARTED before calling self.graph.ainvoke.
If graph.ainvoke completes, the workflow is marked completed and its final state is returned. If an exception escapes the graph, the except Exception handler in ainvoke calls workflow_telemetry.failed("agent_workflow", exc), emits NOC.005 with error and exception_type, and re-raises the same exception. This path does not automatically convert the error into a successful conversational response.
Failures handled inside nodes, such as long-term memory failures, do not trigger the global except. Unhandled errors in routing, agent execution, checkpoint storage, or other stages may reach this boundary and cause a workflow-level failure.
Implementation reference: templates/agent_template_backend/app/workflows/agent_graph.py (persist, ainvoke).
12.11. Operational events and tracing
_node wraps every node execution in self.langgraph_telemetry.node(name, state). Additional mechanisms use workflow spans, semantic events, and the observer for GRL, IC, and NOC. Concrete records in the flow include session.message.rejected_after_end, guardrails.input.completed, guardrails.output.completed, output_supervisor.completed, supervisor.review.completed, long_term_memory.load.failed, long_term_memory.persist.failed, long_term_memory.persist.completed, and workflow_telemetry.failed on workflow-level failure.
Operational errors should be correlated using conversation_key or session_id, together with tenant_id and agent_id when available. A memory or tool failure event does not necessarily indicate overall workflow failure; this depends on whether the component returned a state/result or propagated an exception.
Implementation references: templates/agent_template_backend/app/workflows/agent_graph.py; libs/agent_framework/src/agent_framework/observability/.
13. Integration interfaces
| Interface | Runtime integration |
|---|---|
| LangGraph | State-graph construction, compilation, and execution. |
| Domain agents | Specialized billing, product, order, and support nodes. |
| MCP | Tool selection, parameter mapping, and execution by agents. |
| RAG | Knowledge retrieval incorporated into agent execution. |
| Guardrails | Input and output evaluation. |
| Judges | Evaluation after response generation. |
| Memory | Long-term information loading and writing. |
| Checkpointer | Graph-state persistence. |
| Observability | Per-node telemetry and execution tracing. |
14. Implementation references
libs/agent_framework/src/agent_framework/runtime/templates/agent_template_backend/app/state.pytemplates/agent_template_backend/app/workflows/agent_graph.pytemplates/agent_template_backend/config/agents.yamltemplates/agent_template_backend/config/routing.yamltemplates/agent_template_backend/config/tools.yamlSPEC-004 — MCP GatewaySPEC-005 — GuardrailsSPEC-007 — ObservabilitySPEC-012 — Canonical Contracts