Ajustes conforme relatorio de testes 2026-08-27
This commit is contained in:
@@ -0,0 +1,264 @@
|
||||
### Agent Framework OCI Architecture and Concepts
|
||||
|
||||
### Purpose of this document
|
||||
|
||||
This document **does not replace the root `README_en.md`** and does not repeat the agent-creation tutorial.
|
||||
|
||||
Use:
|
||||
|
||||
- [`README_en.md`](../../../README_en.md) to develop, configure, run, and test an agent end to end;
|
||||
- this document to understand the architecture, responsibility boundaries, components, and where each type of implementation belongs;
|
||||
- the other manuals in this folder to deepen a specific capability or solve a problem.
|
||||
|
||||
The separation is intentional: there is **one main tutorial** and several **specialized reference manuals**.
|
||||
|
||||
### Source of truth
|
||||
|
||||
When documentation diverges, use this order:
|
||||
|
||||
1. code for the version in use;
|
||||
2. `README.md` / `README_en.md` from the same version;
|
||||
3. normative SPECs/SDDs;
|
||||
4. specialized manuals in this folder;
|
||||
5. release notes and `README_old*` only as history.
|
||||
|
||||
### Platform mental model
|
||||
|
||||
Agent Framework OCI should be understood as a layered platform.
|
||||
|
||||
The **framework core** provides reusable, domain-neutral mechanisms: runtime, state, memory, routing, tool integration, guardrails, judges, persistence, observability, and common contracts.
|
||||
|
||||
The **agent** contains what is specific to the use case: intents, prompts, domain rules, specific policies, business workflow, mappings, integrations, and external components that belong to that agent.
|
||||
|
||||
**Gateways** handle cross-cutting ingress, governance, and integration responsibilities. They should not absorb the agent's business logic.
|
||||
|
||||
**MCP Servers** encapsulate tools and integrations with domain or legacy services. The **MCP Gateway** provides centralized catalog and governance for these tools.
|
||||
|
||||
### Main components
|
||||
|
||||
| Component | Main responsibility | Must not contain |
|
||||
|---|---|---|
|
||||
| `libs/agent_framework/` | Generic runtime, contracts, state, memory, routing, guardrails, judges, common integrations | Rule specific to a company or agent |
|
||||
| `templates/agent_template_backend/` | Executable reference for creating agents | Permanent fork of the core |
|
||||
| `apps/agent_gateway/` | Governed ingress, cross-cutting policies, rate limit, authentication, metadata | Business workflow |
|
||||
| `apps/channel_gateway/` | Channel adaptation to the canonical contract | Agent business rule |
|
||||
| `apps/mcp_gateway/` | Catalog, authorization, and centralized tool execution | Conversational logic |
|
||||
| `mcp/servers/` | Integrations and tools by domain | Global agent orchestration |
|
||||
| `evals/` | Certification and regression | Production logic |
|
||||
| `deploy/` | Containers and Kubernetes | Functional rules |
|
||||
|
||||
### Conceptual request flow
|
||||
|
||||
A typical request goes through the following responsibilities:
|
||||
|
||||
```text
|
||||
Canal
|
||||
|
|
||||
v
|
||||
Channel Gateway
|
||||
|
|
||||
v
|
||||
Agent Gateway
|
||||
| governança / autenticação / rate limit / metadata
|
||||
v
|
||||
Backend do agente
|
||||
|
|
||||
+--> Routing / stickiness / intent
|
||||
|
|
||||
+--> Estado / memória / checkpoint
|
||||
|
|
||||
+--> Guardrails / judges
|
||||
|
|
||||
+--> Workflow / políticas transacionais
|
||||
|
|
||||
+--> MCP Gateway
|
||||
|
|
||||
+--> MCP Server A --> sistema legado
|
||||
+--> MCP Server B --> serviço externo
|
||||
+--> MCP Server C --> API de domínio
|
||||
```
|
||||
|
||||
Not every deployment needs to use all components. Composition should follow the agent's needs and the platform contracts.
|
||||
|
||||
### Agent runtime
|
||||
|
||||
The current runtime is based on `AgentRuntimeMixin` and `RuntimeContext`.
|
||||
|
||||
The template imports the runtime through `app.agents.runtime`, which re-exports the framework's official implementation. The goal is to prevent each agent from maintaining its own divergent copy of the runtime.
|
||||
|
||||
Current APIs confirmed in the code include:
|
||||
|
||||
```python
|
||||
AgentRuntimeMixin.get_runtime_context()
|
||||
AgentRuntimeMixin.normalize_tools_by_intent()
|
||||
AgentRuntimeMixin.build_tool_arguments()
|
||||
AgentRuntimeMixin.execute_tools_for_intent()
|
||||
AgentRuntimeMixin.prepare_memory_context()
|
||||
AgentRuntimeMixin.build_messages()
|
||||
AgentRuntimeMixin.transaction_state_patch()
|
||||
AgentRuntimeMixin.transaction_clarification_message()
|
||||
AgentRuntimeMixin.transaction_confirmation_message()
|
||||
AgentRuntimeMixin.build_direct_mcp_answer()
|
||||
```
|
||||
|
||||
These APIs represent runtime capabilities. Developers should prefer them over manually rebuilding the same logic inside each agent.
|
||||
|
||||
### Configuration versus code
|
||||
|
||||
A central framework guideline is that configurable behavior should remain in configuration.
|
||||
|
||||
Examples:
|
||||
|
||||
- agents and metadata: `config/agents.yaml`;
|
||||
- routing: `config/routing.yaml`;
|
||||
- tools: `config/tools.yaml`;
|
||||
- MCP Servers and mappings: corresponding MCP configuration;
|
||||
- LLM profiles: `llm_profiles.yaml`;
|
||||
- policies and extensions: capability-specific configuration files.
|
||||
|
||||
Code should implement mechanisms. YAML/config should select behavior whenever that can be done without compromising security or contracts.
|
||||
|
||||
### Separation between framework and agent
|
||||
|
||||
A change belongs to the **framework** when it introduces a mechanism reusable by different agents.
|
||||
|
||||
Examples:
|
||||
|
||||
- new guardrail SPI;
|
||||
- new rich LLM response contract;
|
||||
- new generic checkpoint capability;
|
||||
- new configurable tool-policy mechanism;
|
||||
- new generic routing strategy.
|
||||
|
||||
A change belongs to the **agent** when it expresses a rule from a domain or company.
|
||||
|
||||
Examples:
|
||||
|
||||
- which charges can be disputed;
|
||||
- a telecom-specific prompt;
|
||||
- VAS rules;
|
||||
- internal company codes;
|
||||
- legacy-service mapping;
|
||||
- specific phraseology.
|
||||
|
||||
If the core needs to import a concrete agent module in order to work, this separation has probably been broken.
|
||||
|
||||
### State, memory, and checkpoint are different concepts
|
||||
|
||||
**Execution state** represents what is happening in the turn and workflow.
|
||||
|
||||
**Conversation memory** preserves conversational context.
|
||||
|
||||
**Long-Term Memory** stores durable facts associated with a business identity.
|
||||
|
||||
**Checkpoint** persists LangGraph state snapshots for resume.
|
||||
|
||||
An old checkpoint must not, by itself, determine which transaction is active. The functional decision must use canonical transaction state.
|
||||
|
||||
### Routing and execution are different responsibilities
|
||||
|
||||
Routing answers: **which agent/intent should handle this message?**
|
||||
|
||||
Execution answers: **what should that agent do now?**
|
||||
|
||||
Route stickiness preserves continuity, but it must not prevent an explicit intent change. During a transaction, expected parameters and valid confirmation take precedence to avoid false intent shifts.
|
||||
|
||||
Full details: [Routing, Stickiness, and Intent Shift](./02_routing_stickiness_and_intent_shift.md).
|
||||
|
||||
### Tools and MCP
|
||||
|
||||
A tool represents an invokable capability.
|
||||
|
||||
The MCP Server implements or exposes that capability.
|
||||
|
||||
The MCP Gateway organizes catalog, authorization, mapping, and centralized execution.
|
||||
|
||||
The agent decides **when** a tool should be used in its flow; the tool/MCP decides **how** to access the corresponding service.
|
||||
|
||||
Full details: [MCP, Tools, Policies, and Parameter Extraction](./04_mcp_integration_tools_and_policies.md).
|
||||
|
||||
### Transactions
|
||||
|
||||
Operations with side effects require different handling from queries.
|
||||
|
||||
The framework provides state, confirmation, policy, and deterministic-workflow mechanisms. Concrete rules remain in the agent.
|
||||
|
||||
The LLM may participate in interpretation and composition, but it must not be the only source of truth for claiming that a critical operation was executed.
|
||||
|
||||
Full details: [Transactional Workflows and State](./03_transaction_workflows_and_state.md).
|
||||
|
||||
### Guardrails and Judges
|
||||
|
||||
Guardrails control or validate behavior during processing.
|
||||
|
||||
Judges evaluate quality, grounding, and other criteria.
|
||||
|
||||
The core provides native mechanisms and extension points. Domain-specific guardrails/judges should be loaded by the agent through configuration, avoiding specific imports inside the framework.
|
||||
|
||||
Full details: [Guardrails, Judges, and Transaction Evaluation](./06_guardrails_judges_and_transaction_evaluation.md).
|
||||
|
||||
### RAG, memory, and tools are not equivalent
|
||||
|
||||
- **RAG** retrieves knowledge.
|
||||
- **Memory** preserves context/facts.
|
||||
- **Tool** executes or queries an external capability.
|
||||
|
||||
Choosing the wrong mechanism creates bugs that are difficult to diagnose. Information that needs to be updated in a system should not be solved only through RAG; a durable customer fact should not depend only on prompt history.
|
||||
|
||||
### Observability as a cross-cutting contract
|
||||
|
||||
Routing, agent, transaction, tool, guardrail, judge, and failure must be correlatable.
|
||||
|
||||
Observability should record what happened, but it must not control business state. Sequence, trace IDs, and labels are diagnostic and audit infrastructure.
|
||||
|
||||
Full details: [Observability, Persistence, and Operational Readiness](./11_observability_persistence_and_operational_readiness.md).
|
||||
|
||||
### Where to place a new feature
|
||||
|
||||
Before implementing, ask these questions:
|
||||
|
||||
1. Is the capability reusable by different agents?
|
||||
2. Is there a domain-specific rule?
|
||||
3. Does it need state across turns?
|
||||
4. Does it produce side effects?
|
||||
5. Does it depend on an external system?
|
||||
6. Should it be configurable?
|
||||
7. Does it need to appear in observability?
|
||||
8. Does it need to be evaluated by a guardrail/judge?
|
||||
|
||||
A reusable feature normally starts in the core and is enabled/configured by the agent. A business rule normally starts in the agent and uses core interfaces.
|
||||
|
||||
### Anti-patterns
|
||||
|
||||
Avoid:
|
||||
|
||||
- importing a concrete agent package inside the core;
|
||||
- duplicating `AgentRuntimeMixin` in every agent;
|
||||
- hardcoding agent, intent, tool, or company names in the runtime;
|
||||
- using an LLM response as proof that an operation was executed;
|
||||
- confusing an old checkpoint with the active transaction;
|
||||
- executing a transactional operation without policy/confirmation when it is required;
|
||||
- coupling an agent directly to dozens of services when MCP Gateway is the intended layer;
|
||||
- creating a new functional document for every bug fix instead of updating the feature manual.
|
||||
|
||||
### Recommended path for a new developer
|
||||
|
||||
1. Read the architectural overview in this document.
|
||||
2. Follow [`README_en.md`](../../../README_en.md) from beginning to end to create and run an agent.
|
||||
3. When you reach a specific capability, use the corresponding specialized manual.
|
||||
4. For failures, start with the [Developer Index](./INDEX_DEVELOPER_GUIDE.md), in the **Search by problem** section.
|
||||
5. Before copying old code, confirm the API/import in the current template and core.
|
||||
|
||||
### Related documents
|
||||
|
||||
- [Main tutorial — README.md](../../../README.md)
|
||||
- [Routing, Stickiness, and Intent Shift](./02_routing_stickiness_and_intent_shift.md)
|
||||
- [Transactional Workflows and State](./03_transaction_workflows_and_state.md)
|
||||
- [MCP, Tools, Policies, and Parameters](./04_mcp_integration_tools_and_policies.md)
|
||||
- [Gateways and Authentication](./05_agent_gateway_mcp_gateway_and_auth.md)
|
||||
- [Guardrails and Judges](./06_guardrails_judges_and_transaction_evaluation.md)
|
||||
- [RAG and BusinessContext](./07_rag_business_context_and_grounding.md)
|
||||
- [Long-Term Memory and Checkpoint](./08_long_term_memory_and_checkpoint.md)
|
||||
- [LLM Rich Response](./09_llm_rich_response_reasoning.md)
|
||||
- [Performance, Cache, and Async Runtime](./10_performance_cache_and_async_runtime.md)
|
||||
- [Observability and Operational Readiness](./11_observability_persistence_and_operational_readiness.md)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,638 @@
|
||||
### Transactional Workflows and State
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **transaction state, parameter collection, confirmation, pause/resume, and operational evidence**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
Transaction state, parameter collection, confirmation, pause/resume, and operational evidence.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### Transactional Workflows, Multi-turn State, and Resume
|
||||
|
||||
Implementation guide for multi-step operations, canonical transaction-state source, confirmation, parameter merge, pause/resume, operational evidence, and routing interaction.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Multi-turn transaction-state guide
|
||||
|
||||
> Content consolidated from `docs/TRANSACTION_STATE_DEVELOPER_GUIDE.md`.
|
||||
|
||||
This document defines the operational contract for multi-turn transactions in Agent Framework OCI. It is normative for hosts and templates that use `AgentRuntime`, LangGraph checkpointing, and transactional tools.
|
||||
|
||||
### 1. Goal
|
||||
|
||||
A transaction can span several turns. Example:
|
||||
|
||||
```text
|
||||
Usuário: quero cancelar o pedido
|
||||
Framework: informe o número do pedido
|
||||
Usuário: PED-1001
|
||||
Framework: confirma o cancelamento?
|
||||
Usuário: sim
|
||||
Framework: executa a tool
|
||||
```
|
||||
|
||||
The framework must preserve the transaction across all these turns without depending on LLM reclassification, keyword routing, or re-extraction of parameters that have already been obtained.
|
||||
|
||||
### 2. Canonical transaction-state source
|
||||
|
||||
The canonical state for the in-progress transaction is `active_transaction`.
|
||||
|
||||
```python
|
||||
active_transaction: dict[str, Any]
|
||||
last_transaction: dict[str, Any]
|
||||
```
|
||||
|
||||
Every `AgentState` used by a host that enables multi-turn transactions **MUST** declare both fields. Because LangGraph uses the state schema for persistence/checkpointing, a field created only dynamically by the runtime is not a safe durable contract.
|
||||
|
||||
Minimum example:
|
||||
|
||||
```python
|
||||
from typing import Any, TypedDict
|
||||
|
||||
class AgentState(TypedDict, total=False):
|
||||
# ...campos normais...
|
||||
selected_tool_call: dict[str, Any]
|
||||
pending_tool_call: dict[str, Any]
|
||||
active_transaction: dict[str, Any]
|
||||
last_transaction: dict[str, Any]
|
||||
transaction_status: str
|
||||
missing_parameters: list[str]
|
||||
confirmation_required: bool
|
||||
confirmation_received: bool
|
||||
```
|
||||
|
||||
### 3. Role of each field
|
||||
|
||||
| Field | Role | Rule |
|
||||
|---|---|---|
|
||||
| `active_transaction` | Canonical source of the active transaction | Must survive checkpoint/resume while the transaction is active. |
|
||||
| `last_transaction` | Snapshot of the last terminal transaction | Used for audit, evidence, and controlled continuity; it does not automatically reactivate the transaction. |
|
||||
| `transaction_status` | Current logical state | E.g. `COLLECTING_PARAMETERS`, `AWAITING_CONFIRMATION`, `COMPLETED`, `CANCELLED`, `OUT_OF_SCOPE`. |
|
||||
| `missing_parameters` | Parameters still required | Must reflect canonical transaction state, not only the current message. |
|
||||
| `selected_tool_call` | Auxiliary/compatibility state | Must not replace `active_transaction` as the canonical source. |
|
||||
| `pending_tool_call` | Auxiliary/compatibility state | May be used for compatibility, but not as the primary latch. |
|
||||
| `next_state` | Workflow routing guidance | Helps keep the correct node/agent during collection/confirmation. |
|
||||
| `transaction_pre_validation` | Pre-validation evidence | Preserves validation results before confirmation/execution. |
|
||||
| `transaction_evidence` | Execution evidence | Preserves transaction results and execution trail. |
|
||||
|
||||
### 4. Recommended lifecycle
|
||||
|
||||
```text
|
||||
IDLE
|
||||
↓ intenção transacional
|
||||
COLLECTING_PARAMETERS
|
||||
↓ parâmetros completos
|
||||
PRE_VALIDATION (quando configurado)
|
||||
↓ elegível
|
||||
AWAITING_CONFIRMATION
|
||||
↓ confirmação positiva
|
||||
EXECUTING
|
||||
↓
|
||||
COMPLETED
|
||||
```
|
||||
|
||||
Alternative terminal outcomes:
|
||||
|
||||
```text
|
||||
CANCELLED
|
||||
OUT_OF_SCOPE
|
||||
FAILED
|
||||
```
|
||||
|
||||
The runtime may represent some phases internally without a separate public `transaction_status`. The requirement is to preserve the latch and not lose arguments already collected.
|
||||
|
||||
### 5. Incremental parameter merge
|
||||
|
||||
A later response must complement the existing transaction, never recreate it only from the current text.
|
||||
|
||||
```python
|
||||
existing = dict((state.get("active_transaction") or {}).get("arguments") or {})
|
||||
new_values = {"valor": "71.99"}
|
||||
arguments = {**existing, **new_values}
|
||||
```
|
||||
|
||||
Expected example:
|
||||
|
||||
```text
|
||||
Turno 1: subject = "TIM CTRL Redes Sociais 8.0"
|
||||
Turno 2: valor = "71.99"
|
||||
Resultado: subject + valor permanecem disponíveis
|
||||
```
|
||||
|
||||
### 6. Routing precedence during a transaction
|
||||
|
||||
When an `active_transaction` exists in `COLLECTING_PARAMETERS`, the message must first be evaluated as a possible answer to the pending parameters.
|
||||
|
||||
Normative precedence:
|
||||
|
||||
1. pending parameter clearly filled → continue the transaction;
|
||||
2. explicit cancellation/abandonment → cancel the transaction;
|
||||
3. unequivocal new intent → interrupt the transaction and route;
|
||||
4. generic keyword from the same domain/agent → **do not** interrupt the transaction;
|
||||
5. ambiguous message → keep the transaction and clarify.
|
||||
|
||||
Examples:
|
||||
|
||||
| Current state | Message | Correct result |
|
||||
|---|---|---|
|
||||
| `retail_order_cancel`, missing `order_id` | `PED-1001` | Continue cancellation and fill `order_id`. |
|
||||
| `retail_order_cancel`, missing `order_id` | `o pedido é o PED-1001` | Continue cancellation; `pedido` must not become tracking. |
|
||||
| dispute, missing `valor` | `R$ 71,99` | Continue dispute and fill `valor`. |
|
||||
| cancellation pending | `esquece, quero ver minha fatura` | Explicit interruption allowed. |
|
||||
| cancellation pending | `quero rastrear pedido` | Unequivocal shift to tracking allowed. |
|
||||
|
||||
### 7. Checkpoint and resume
|
||||
|
||||
Before running normal routing, the host must restore the checkpoint using the same conversation identity (`tenant_id`, `agent_id`, `session_id`/`conversation_key` according to the host contract).
|
||||
|
||||
After restoration:
|
||||
|
||||
```text
|
||||
active_transaction existe
|
||||
↓
|
||||
status ativo?
|
||||
↓ sim
|
||||
retomar a transação antes de keyword routing / continuity LLM
|
||||
```
|
||||
|
||||
A `COLLECTING_PARAMETERS` state without `active_transaction` must be treated as a state inconsistency and observed/diagnosed; it must not silently restart the tool from the current message.
|
||||
|
||||
### 8. What belongs to the framework and what belongs to the agent
|
||||
|
||||
Framework:
|
||||
|
||||
- latch persistence;
|
||||
- argument merge;
|
||||
- collection/confirmation states;
|
||||
- resume precedence;
|
||||
- deterministic confirmation;
|
||||
- idempotency and evidence;
|
||||
- checkpoint/resume.
|
||||
|
||||
Agent:
|
||||
|
||||
- domain-tool definitions;
|
||||
- required parameters and domain messages;
|
||||
- domain-specific eligibility rules;
|
||||
- domain-specific pre-validation, when applicable;
|
||||
- final customer response.
|
||||
|
||||
The agent must not implement a second transactional engine in parallel with `AgentRuntime`.
|
||||
|
||||
### 9. Checklist for new hosts/templates
|
||||
|
||||
- [ ] `AgentState` declares `active_transaction`.
|
||||
- [ ] `AgentState` declares `last_transaction`.
|
||||
- [ ] `transaction_status` and `missing_parameters` are part of state when used.
|
||||
- [ ] The host uses checkpointing compatible with the state schema.
|
||||
- [ ] The same `conversation_key` is used across turns of the same conversation.
|
||||
- [ ] Previously collected parameters are merged with new values.
|
||||
- [ ] Parameter answers take precedence over generic keyword routing.
|
||||
- [ ] Explicit intent changes remain possible.
|
||||
- [ ] The agent uses `transaction_state_patch(state)` when returning transactional responses if the template requires it.
|
||||
- [ ] Multi-turn tests exist for collection, confirmation, interruption, and resume.
|
||||
|
||||
### 10. Minimum regression tests
|
||||
|
||||
```text
|
||||
A. cancelamento de pedido
|
||||
1. "quero cancelar pedido"
|
||||
2. "o pedido é o PED-1001"
|
||||
Esperado: continua retail_order_cancel; não vira retail_order_tracking.
|
||||
|
||||
B. contestação
|
||||
1. "não contratei TIM CTRL Redes Sociais 8.0"
|
||||
2. "R$ 71,99"
|
||||
Esperado: subject e valor chegam juntos à pre-validation.
|
||||
|
||||
C. interrupção explícita
|
||||
1. iniciar transação e deixar parâmetro pendente
|
||||
2. "esquece, quero ver minha fatura"
|
||||
Esperado: transação é interrompida e nova intenção é roteada.
|
||||
|
||||
D. checkpoint/resume
|
||||
1. iniciar transação
|
||||
2. persistir/checkpoint
|
||||
3. reconstruir execução usando a mesma conversation_key
|
||||
4. fornecer o parâmetro faltante
|
||||
Esperado: active_transaction é restaurado e concluído sem reiniciar a tool.
|
||||
```
|
||||
|
||||
### 11. Anti-patterns
|
||||
|
||||
- rebuilding the transaction only from the last message;
|
||||
- using `selected_tool_call` as the only latch source;
|
||||
- removing `active_transaction` from `AgentState` because it appears redundant;
|
||||
- allowing a generic keyword such as `pedido` to interrupt `order_id` collection;
|
||||
- storing parameters only in local node variables;
|
||||
- duplicating transactional confirmation in the agent prompt;
|
||||
- clearing the latch before terminal state.
|
||||
|
||||
### 12. Project references
|
||||
|
||||
- `specs/SPEC-002-Agent-Runtime.md`
|
||||
- `specs/SPEC-010-Agent-Development.md`
|
||||
- `templates/agent_template_backend/app/state.py`
|
||||
- `libs/agent_framework/src/agent_framework/runtime/agent_runtime.py`
|
||||
- `libs/agent_framework/src/agent_framework/routing/enterprise_router.py`
|
||||
- `Tuning-Performance/Deterministic_Transactional_Workflow/`
|
||||
- `Tuning-Performance/Transaction_Pre_Validation/`
|
||||
- `Tuning-Performance/Transaction_Evidence/`
|
||||
|
||||
### Transactional workflow engine architectural decision
|
||||
|
||||
> Content consolidated from `docs/ADR_TRANSACTIONAL_WORKFLOW_ENGINE.md`.
|
||||
|
||||
### Decision
|
||||
|
||||
Add an optional deterministic execution capability based on LangGraph to the framework. The engine is generic; YAML definitions and domain actions remain in the agents.
|
||||
|
||||
### Rationale
|
||||
|
||||
Multi-step operations with side effects must not depend on the LLM to select the critical sequence. The solution reduces tokens, latency, and variability, while improving auditability, testing, and versioning.
|
||||
|
||||
### Compatibility
|
||||
|
||||
`execution.mode` defaults to `direct_tool`. Existing projects continue to use MCP directly. Workflow adoption is explicit per tool and may be controlled through `ENABLE_TRANSACTIONAL_WORKFLOWS`.
|
||||
|
||||
### Scope limits of this delivery
|
||||
|
||||
The foundation includes validation, file-based versioning, registry, sync/async execution, conditions, per-node retry, graph cache, and a policy adapter. Enterprise execution-record persistence, compensation/Saga, scope authorization, and workflow-specific IC/NOC emission must be connected to the abstractions available in each deployment before use in critical financial transactions.
|
||||
|
||||
### Deterministic workflow implementation
|
||||
|
||||
> Content consolidated from `Documentacao/IMPLEMENTACAO_WORKFLOWS_TRANSACIONAIS.md`.
|
||||
|
||||
### Delivery
|
||||
|
||||
An optional capability was added to `agent_framework_oci` to execute multi-step transactions as deterministic workflows compiled into LangGraph.
|
||||
|
||||
### New module
|
||||
|
||||
`libs/agent_framework/src/agent_framework/workflows/`
|
||||
|
||||
- `models.py`: Pydantic contracts and structural validation;
|
||||
- `repository.py`: active-version resolution and immutable YAML reading;
|
||||
- `registry.py`: decoupled registration of sync/async actions;
|
||||
- `runtime.py`: StateGraph compilation, cache, and execution;
|
||||
- `tool_executor.py`: integration with tool policy;
|
||||
- `__init__.py`: public API.
|
||||
|
||||
### Expanded policy
|
||||
|
||||
`ToolPolicy` now accepts:
|
||||
|
||||
```yaml
|
||||
execution:
|
||||
mode: direct_tool | workflow | agent
|
||||
workflow: nome_do_workflow
|
||||
version: active | 1
|
||||
```
|
||||
|
||||
The default remains `direct_tool`, preserving compatibility.
|
||||
|
||||
### Configuration
|
||||
|
||||
The following were added:
|
||||
|
||||
- `ENABLE_TRANSACTIONAL_WORKFLOWS=false`;
|
||||
- `WORKFLOWS_PATH=./workflows`.
|
||||
|
||||
### Template
|
||||
|
||||
Includes a complete order-return example with:
|
||||
|
||||
- confirmation and required fields from policy;
|
||||
- versioned workflow YAML;
|
||||
- domain actions in the backend;
|
||||
- deterministic branching based on validation result.
|
||||
|
||||
### Validation performed
|
||||
|
||||
- `tests/unit/test_tool_policies.py`: 4 tests passed;
|
||||
- Python compilation for framework, template, and new tests: passed;
|
||||
- the new LangGraph functional test was created but could not be run in this container because `langgraph` is not installed in the environment. The dependency is already declared in the framework `pyproject.toml`.
|
||||
|
||||
### Scope and safety
|
||||
|
||||
This delivery creates the engine and policy integration. For critical production operations, it is still necessary to connect:
|
||||
|
||||
- persistent execution store;
|
||||
- business idempotency in actions/APIs;
|
||||
- scope authorization;
|
||||
- workflow-specific IC/NOC telemetry;
|
||||
- compensation/Saga where applicable;
|
||||
- enterprise timeout and retry strategy.
|
||||
|
||||
These items are explicitly documented to avoid the false impression that retry by itself guarantees transactional safety.
|
||||
|
||||
### Parameter-collection precedence
|
||||
|
||||
> Content consolidated from `FIX_TRANSACTION_PARAMETER_PRECEDENCE.md`.
|
||||
|
||||
This correction removes hardcoded textual extraction of transactional parameters and collects `policy.requires` through a generic LLM extractor.
|
||||
|
||||
### Precedence rule
|
||||
|
||||
While an active transaction exists, the framework handles the turn in this order:
|
||||
|
||||
```text
|
||||
ACTIVE_TRANSACTION
|
||||
|
|
||||
+-- COLLECTING_PARAMETERS
|
||||
| |
|
||||
| +-- LLM tenta extrair SOMENTE os parâmetros ainda pendentes
|
||||
| |
|
||||
| +-- extraiu >= 1 ?
|
||||
| |
|
||||
| +-- SIM -> continua a transação; NÃO avalia intent_shift
|
||||
| |
|
||||
| +-- NÃO -> libera EnterpriseRouter para avaliar intent_shift
|
||||
|
|
||||
+-- AWAITING_CONFIRMATION
|
||||
|
|
||||
+-- reconhece confirmação/rejeição explícita
|
||||
|
|
||||
+-- reconheceu ?
|
||||
|
|
||||
+-- SIM -> continua/cancela a transação; NÃO avalia intent_shift
|
||||
|
|
||||
+-- NÃO -> libera EnterpriseRouter para avaliar intent_shift
|
||||
```
|
||||
|
||||
### TransactionParameterExtractor
|
||||
|
||||
New component:
|
||||
|
||||
`libs/agent_framework/src/agent_framework/runtime/transaction_parameters.py`
|
||||
|
||||
Textual extraction of business parameters is performed exclusively by the LLM. The component receives:
|
||||
|
||||
- name of the active tool/transaction;
|
||||
- currently pending parameters;
|
||||
- already known arguments;
|
||||
- schema/types declared in `tools.yaml` when available;
|
||||
- tool description;
|
||||
- current user message.
|
||||
|
||||
It does not know domain names such as `order_id`, `reason`, `subject`, `valor`, TIM, or retail. There is no regex for business entities.
|
||||
|
||||
The LLM can interpret, for example:
|
||||
|
||||
- `PED-1001` when only one compatible parameter is pending;
|
||||
- `o pedido é PED-1001`;
|
||||
- `PED-1001, desisti da compra`, filling two parameters in the same turn;
|
||||
- answers with the parameter name followed by the value;
|
||||
- answers containing only the value, when semantically unequivocal.
|
||||
|
||||
When in doubt, the prompt instructs the model to return `null`. A new request must not be transformed into a parameter value.
|
||||
|
||||
### Separation of responsibilities
|
||||
|
||||
`tool_policies.yaml` remains the source of truth for `requires`.
|
||||
|
||||
`tools.yaml` may provide types through `args_schema` and the tool description to improve interpretation without introducing domain-specific code.
|
||||
|
||||
`mcp_parameter_mapping.yaml` remains responsible for auxiliary parameters/MCP contract. Mapper strategies are explicitly excluded for fields present in `policy.requires`, so MCP extraction is not mixed with transactional collection.
|
||||
|
||||
The `EnterpriseRouter` uses the same LLM extractor only as a precedence *probe*. If at least one pending parameter is found, the turn remains in the transactional state. Extracted values are placed in decision metadata and reused by the runtime, avoiding a second LLM call in the same turn.
|
||||
|
||||
### LLM profile
|
||||
|
||||
The following was added to the templates:
|
||||
|
||||
```yaml
|
||||
transaction_parameter_extraction:
|
||||
provider: oci_openai
|
||||
model: openai.gpt-4.1-mini
|
||||
temperature: 0
|
||||
max_tokens: 500
|
||||
timeout_seconds: 8
|
||||
```
|
||||
|
||||
Generation/component:
|
||||
|
||||
- `llm.transaction_parameter_extraction`
|
||||
- `transaction_parameter_extraction`
|
||||
|
||||
### State cleanup
|
||||
|
||||
On `intent_shift`, the abandoned transaction's `transaction_pre_validation` is removed so it does not contaminate the new transaction. The pre-validation result remains preserved while it belongs to its own transaction for audit purposes.
|
||||
|
||||
### Tests added
|
||||
|
||||
`tests/test_transaction_parameter_llm_precedence.py`
|
||||
|
||||
Coverage:
|
||||
|
||||
1. two parameters extracted in the same turn;
|
||||
2. one filled parameter takes precedence over a keyword that would indicate another intent;
|
||||
3. no parameter found releases `intent_shift`;
|
||||
4. absence of the old hardcoded `_extract_action_arguments()`;
|
||||
5. `sim` confirmation takes precedence over intent shift.
|
||||
|
||||
### Transaction/intent loop fix
|
||||
|
||||
> Content consolidated from `FIX_TRANSACTION_INTENT_LOOP.md`.
|
||||
|
||||
Correction applied on 2026-08-20 to prevent a session from getting stuck in `COLLECTING_PARAMETERS` or `AWAITING_CONFIRMATION` when the user explicitly changes subject.
|
||||
|
||||
### Corrected behavior
|
||||
|
||||
Before:
|
||||
|
||||
1. a transaction entered `COLLECTING_PARAMETERS`;
|
||||
2. `next_state` forced the same agent through `state_policies`;
|
||||
3. every following message was treated as an attempt to fill the missing parameter;
|
||||
4. a new intent such as `quais sao meus servicos` remained trapped in the previous flow.
|
||||
|
||||
Now:
|
||||
|
||||
- the `EnterpriseRouter` checks for an explicit intent change before applying the state lock;
|
||||
- explicit keyword has priority;
|
||||
- when necessary, the LLM router can detect a change with confidence >= `router.confidence_threshold`;
|
||||
- the decision receives `metadata.transaction_interruption=intent_shift`;
|
||||
- the runtime closes the pending transaction as `CANCELLED`, clears `next_state`, parameters, and latches, and proceeds with the new intent;
|
||||
- explicit cancellations such as `cancele essa operação anterior` also work during `COLLECTING_PARAMETERS`.
|
||||
|
||||
### Tests added
|
||||
|
||||
- intent change during `COLLECTING_PARAMETERS`;
|
||||
- short/low-confidence answer remains in the transaction;
|
||||
- explicit cancellation during parameter collection;
|
||||
- cleanup of transactional state before executing the new intent.
|
||||
|
||||
Focused tests: 19 passed.
|
||||
|
||||
### Operational execution evidence
|
||||
|
||||
> Content consolidated from `docs/TRANSACTION_OPERATIONAL_EVIDENCE_FIX.md`.
|
||||
|
||||
### Problem
|
||||
|
||||
A confirmed transactional tool result was available only in the execution turn. On a later read-only turn, conversational memory could still mention the prior transaction (for example, a cancellation protocol), while the groundedness judge received only the current MCP results. This could classify a factually correct follow-up as unsupported.
|
||||
|
||||
### Fix
|
||||
|
||||
The framework now records completed/failed transactional tool outcomes as bounded operational evidence in LangGraph state/checkpoint (`transaction_evidence`). This is operational state, not Long Term Memory.
|
||||
|
||||
For each new turn, the runtime correlates previous transaction evidence with the current resource using generic identifiers (`*_id`, `order_id`, `invoice_id`, `asset_id`, `resource_key`, etc.). Only relevant evidence is materialized as `relevant_transaction_evidence`.
|
||||
|
||||
The same relevant evidence is:
|
||||
|
||||
- injected into the answering LLM prompt;
|
||||
- merged with current MCP results for groundedness judges;
|
||||
- exposed in response metadata as `transaction_evidence` for diagnostics;
|
||||
- emitted with the completion telemetry event.
|
||||
|
||||
The history is bounded to the 10 most recent transaction outcomes, and at most 5 correlated entries are injected for a turn.
|
||||
|
||||
### Expected retail example
|
||||
|
||||
1. `cancelar_pedido(PED-1001)` returns protocol `CANCEL-2026-001`.
|
||||
2. The result is persisted as transaction evidence.
|
||||
3. The next `consultar_pedido(PED-1001)` returns `EM_TRANSPORTE`.
|
||||
4. The answering agent and groundedness judge receive both the current order result and the prior cancellation evidence.
|
||||
5. A response that mentions `CANCEL-2026-001` is grounded rather than treated as an unsupported claim.
|
||||
|
||||
### Integrated Backend/MCP validation
|
||||
|
||||
> Content consolidated from `Documentacao/VALIDACAO_TRANSACIONAL_BACKEND_MCP.md`.
|
||||
|
||||
### Implemented corrections
|
||||
|
||||
- `mcp_tools` is treated as an allowlist, not as an automatic execution list.
|
||||
- `read_only` tools remain available for context enrichment.
|
||||
- Only one transactional tool compatible with the request is selected.
|
||||
- `require_confirmation: true` creates `pending_tool_call` and `AWAITING_CONFIRMATION`.
|
||||
- The confirmation turn executes the pending call with `confirmed: true`.
|
||||
- State exposes `selected_tool_call`, `tool_policy_result`, `confirmation_required`, `confirmation_received`, and `transaction_status`.
|
||||
- `reason` was standardized across catalog, mapping, and Retail FastMCP.
|
||||
- Orders `123` and `PED-ENTREGUE` return status `ENTREGUE` for positive tests.
|
||||
- The generic keyword `produto` was removed from the Telecom intent so it does not capture Retail returns.
|
||||
- `Normal` and `Route_Stickness` templates in `Tuning-Performance` were updated.
|
||||
|
||||
### Recommended test
|
||||
|
||||
1. `Quero devolver o pedido 123 porque me arrependi da compra.`
|
||||
2. Expected: `transaction_status=AWAITING_CONFIRMATION`, without executing `solicitar_devolucao`.
|
||||
3. `Sim, confirmo a devolução.`
|
||||
4. Expected: `transaction_status=COMPLETED` and a single execution of `solicitar_devolucao`.
|
||||
|
||||
### Automated result
|
||||
|
||||
```text
|
||||
7 passed
|
||||
```
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `docs/TRANSACTION_STATE_DEVELOPER_GUIDE.md`
|
||||
- `docs/ADR_TRANSACTIONAL_WORKFLOW_ENGINE.md`
|
||||
- `Documentacao/IMPLEMENTACAO_WORKFLOWS_TRANSACIONAIS.md`
|
||||
- `FIX_TRANSACTION_PARAMETER_PRECEDENCE.md`
|
||||
- `FIX_TRANSACTION_INTENT_LOOP.md`
|
||||
- `docs/TRANSACTION_OPERATIONAL_EVIDENCE_FIX.md`
|
||||
- `Documentacao/VALIDACAO_TRANSACIONAL_BACKEND_MCP.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
|
||||
|
||||
## Canonical resolution and domain revalidation before execution
|
||||
|
||||
When pre-validation resolves a user reference to a canonical entity, the framework **must not blindly overwrite the parameter and execute the originally selected tool**. The contract keeps requested, resolved and execution values distinct.
|
||||
|
||||
A domain validator may return `transaction_decision` with `resolved_arguments`, `target_tool`, `action_changed`, `requires_reconfirmation`, and an optional customer-facing `confirmation_message`.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- **Framework:** preserve the requested arguments, apply only canonical arguments declared by the validator, update the transaction to the effective `target_tool`, honor reconfirmation, and retain the decision in pre-validation evidence.
|
||||
- **Agent/domain:** decide business class, policy and effective tool. The framework must not know rules such as “Youtube Premium is strategic”.
|
||||
- **MCP/backend:** execute the final operation chosen by the domain.
|
||||
|
||||
If canonicalization does not change the action, the current tool may remain valid. If entity resolution changes business class/policy/tool, domain revalidation must happen **before confirmation and execution**. Ambiguous or low-confidence resolution must request clarification instead of silently promoting a candidate.
|
||||
|
||||
### Troubleshooting: resolved_subject is correct but execution receives the original text
|
||||
|
||||
If pre-validation records `resolved_subject="Youtube Premium"` while execution still receives `subject="youtube"`, verify that the validator returns `transaction_decision.resolved_arguments` and that the runtime applies the decision before freezing `pending_tool_call` / `confirmation_snapshot`. If the canonical entity is correct but the final tool is wrong, inspect `transaction_decision.target_tool`; that business reclassification belongs to the domain validator, not to the framework.
|
||||
|
||||
For domains that expose an authoritative business classification in backend detail, revalidation should use that evidence before aggregated categories. In Contas, for example, `invoice_detail.parsed_content` preserves `classe=avulso|estrategico|bundle`, while `billing_analysis` may group the same item into broader sections such as `streaming` or partner services. Canonical entity discovery may use any authorized evidence, but the **business decision** should prioritize the source that preserves the domain classification. If classification evidence conflicts, do not silently change the action; preserve the current operation or request clarification according to the agent policy.
|
||||
|
||||
|
||||
## Semantic transactional confirmation: SIM / NAO / CONTINUAR
|
||||
|
||||
Transactions in `AWAITING_CONFIRMATION` use two layers, in this order:
|
||||
|
||||
1. **Deterministic parser** for explicit confirmations/rejections (`sim`, `não`, `confirmo`, `pode fazer`, etc.). This remains the cheapest and safest path and **does not call an LLM**.
|
||||
2. **LLM semantic fallback** only when the deterministic parser is inconclusive. The fallback reuses the same declarative semantic-classifier engine used by paused workflow `expected_input`, injecting the pending prompt, recent context related to the same topic, and the current user utterance.
|
||||
|
||||
Configuration lives in `config/routing.yaml` under `router.transaction_confirmation.semantic_fallback`:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
transaction_confirmation:
|
||||
semantic_fallback:
|
||||
enabled: true
|
||||
allowed_values: [SIM, NAO, CONTINUAR]
|
||||
confirm_values: [SIM]
|
||||
reject_values: [NAO]
|
||||
continue_values: [CONTINUAR]
|
||||
include_relevant_context: true
|
||||
profile_name: router
|
||||
prompt: |
|
||||
Allowed classes: {{ allowed_values }}
|
||||
Pending prompt:
|
||||
{{ pending_prompt }}
|
||||
Relevant context:
|
||||
{{ relevant_conversation_context }}
|
||||
Current user input:
|
||||
{{ user_input }}
|
||||
```
|
||||
|
||||
`SIM` means an unambiguous acceptance, `NAO` an unambiguous rejection, and `CONTINUAR` means the utterance does not safely confirm or reject the pending action. Example: after `Você confirma o cancelamento do serviço Tamboro Mensal?`, the reply `isso mesmo, pode confirmar` can be classified as `SIM` without hardcoding that exact sentence.
|
||||
|
||||
When semantic confirmation succeeds, the router records:
|
||||
|
||||
```json
|
||||
{
|
||||
"transaction_turn_consumed": true,
|
||||
"transaction_confirmation_decision": "confirm",
|
||||
"transaction_confirmation_source": "semantic"
|
||||
}
|
||||
```
|
||||
|
||||
`AgentRuntime` reuses this routed decision instead of re-running the deterministic parser. The change is additive: existing explicit yes/no inputs continue through the deterministic path with no extra LLM call. Semantic generations are named `transaction.confirmation.semantic_classifier` for observability.
|
||||
|
||||
### Durable interrupt compatibility in pause/resume
|
||||
|
||||
The runtime does not use `snapshot.next` alone to decide whether a workflow is paused. A truthy `next` may represent LangGraph helper work, including framework-generated synthetic nodes such as `__pause` and `__continue`.
|
||||
|
||||
A pause is recognized only from a real interrupt. Depending on the LangGraph/checkpointer version, that interrupt may be exposed through `task.interrupts` or persisted in `snapshot.values["__interrupt__"]`. The runtime supports both shapes and deduplicates the payload when both are present.
|
||||
|
||||
This prevents two false diagnoses:
|
||||
|
||||
- treating `snapshot.next` as `PAUSED` when no real interrupt exists;
|
||||
- treating `next=("<node>__pause",)` as invalid pending work when the real interrupt is persisted under `__interrupt__`.
|
||||
|
||||
For workflows using `expected_input.semantic_classifier`, internal tokens such as `SIM`, `NAO`, and `CONTINUAR` remain resume control values and must not be confused with customer-facing output.
|
||||
@@ -0,0 +1,821 @@
|
||||
### MCP, Tools, Policies, and Parameter Extraction
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **tools, MCP Servers, mappings, read-only/transactional policies, and parameter extraction**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
Tools, MCP servers, mappings, read-only/transactional policies, and parameter extraction.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### MCP Integration, Tools, Policies, and Parameter Extraction
|
||||
|
||||
Development manual for integrating MCP Servers, registering tools, isolating tools by agent, configuring read-only/transactional policies, confirmation, and contextual parameter extraction.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Complete MCP Server integration manual
|
||||
|
||||
> Content consolidated from `Documentacao/Manual_Integracao_MCP_Servers_Agent_Framework.docx`.
|
||||
|
||||
MCP Server Integration Manual
|
||||
Multi-Agent Framework - Router, Supervisor, Tools, and External Servers
|
||||
This document explains MCP concepts, how the current project integrates MCP servers, how to start the example Telecom and Retail servers, how to configure tools per agent, and how to evolve the implementation toward a solution that more closely follows the official MCP standard. The goal is to serve as a development, local-operations, and container/OCI deployment guide.
|
||||
|
||||
### MCP concepts
|
||||
|
||||
MCP stands for Model Context Protocol. It defines a standardized way for AI applications to access external context, tools, and capabilities from systems outside the model. Instead of putting integrations directly into the prompt or agent, MCP separates responsibilities: the agent decides what it needs, and an MCP server exposes tools, resources, and prompts in a controlled way.
|
||||
In the official standard, MCP uses JSON-RPC messages and defines transports such as stdio and Streamable HTTP. The current project uses a simplified HTTP implementation to make understanding and local testing easier, with REST endpoints `/mcp/tools/list` and `/mcp/tools/call`. This is appropriate for tutorials and prototyping, but it can later evolve to an official MCP client.
|
||||
|
||||
### How the current project organizes MCP
|
||||
|
||||
The relevant project structure is:
|
||||
```
|
||||
projeto_multi_agent_isolado/
|
||||
agent_framework/
|
||||
src/agent_framework/mcp/
|
||||
client.py
|
||||
models.py
|
||||
registry.py
|
||||
tool_router.py
|
||||
|
||||
agent_template_backend/
|
||||
config/
|
||||
mcp_servers.yaml
|
||||
mcp_servers.docker.yaml
|
||||
tools.yaml
|
||||
mcp_parameter_mapping.yaml
|
||||
app/
|
||||
main.py
|
||||
workflows/agent_graph.py
|
||||
|
||||
mcp_servers/
|
||||
telecom_mcp_server/
|
||||
main.py
|
||||
requirements.txt
|
||||
Dockerfile
|
||||
retail_mcp_server/
|
||||
main.py
|
||||
requirements.txt
|
||||
Dockerfile
|
||||
|
||||
scripts/
|
||||
run_mcp_servers.sh
|
||||
docker-compose.yml
|
||||
```
|
||||
|
||||
### Main components
|
||||
|
||||
|
||||
### Simplified HTTP contract used by the project
|
||||
|
||||
```
|
||||
GET /mcp/tools/list
|
||||
POST /mcp/tools/call
|
||||
|
||||
Payload de chamada:
|
||||
{
|
||||
"tool_name": "consultar_fatura",
|
||||
"arguments": {
|
||||
"msisdn": "11999999999",
|
||||
"invoice_id": "INV-001"
|
||||
}
|
||||
}
|
||||
|
||||
Resposta esperada:
|
||||
{
|
||||
"ok": true,
|
||||
"result": { ... },
|
||||
"metadata": {
|
||||
"server": "telecom",
|
||||
"tool": "consultar_fatura"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### How to start the example MCP servers
|
||||
|
||||
The project includes two example MCP servers: Telecom and Retail. They are independent FastAPI apps. The Telecom server runs on port 8100 and exposes tools such as `consultar_fatura`, `consultar_pagamentos`, `consultar_plano`, and `listar_servicos`. The Retail server runs on port 8200 and exposes tools such as `consultar_pedido`, `consultar_entrega`, `solicitar_troca`, and `solicitar_devolucao`.
|
||||
|
||||
### Local startup through the script
|
||||
|
||||
```
|
||||
cd projeto_multi_agent_isolado
|
||||
bash ./scripts/run_mcp_servers.sh
|
||||
```
|
||||
The script creates a venv in the root directory, installs the MCP-server dependencies, and starts both uvicorn processes in the background:
|
||||
```
|
||||
Telecom MCP: http://localhost:8100
|
||||
Retail MCP: http://localhost:8200
|
||||
```
|
||||
|
||||
### Manual startup of Telecom MCP
|
||||
|
||||
```
|
||||
cd projeto_multi_agent_isolado
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r mcp_servers/telecom_mcp_server/requirements.txt
|
||||
uvicorn --app-dir mcp_servers/telecom_mcp_server main:app --host 0.0.0.0 --port 8100
|
||||
```
|
||||
|
||||
### Manual startup of Retail MCP
|
||||
|
||||
```
|
||||
cd projeto_multi_agent_isolado
|
||||
source .venv/bin/activate
|
||||
pip install -r mcp_servers/retail_mcp_server/requirements.txt
|
||||
uvicorn --app-dir mcp_servers/retail_mcp_server main:app --host 0.0.0.0 --port 8200
|
||||
```
|
||||
|
||||
### Startup with Docker Compose
|
||||
|
||||
```
|
||||
cd projeto_multi_agent_isolado
|
||||
docker compose up --build
|
||||
```
|
||||
In Docker Compose, the backend uses `mcp_servers.docker.yaml` because, inside the compose network, localhost would point to the backend container itself. Therefore the endpoints use service names: `telecom-mcp` and `retail-mcp`.
|
||||
```
|
||||
services:
|
||||
telecom-mcp:
|
||||
ports:
|
||||
- "8100:8100"
|
||||
|
||||
retail-mcp:
|
||||
ports:
|
||||
- "8200:8200"
|
||||
|
||||
backend:
|
||||
environment:
|
||||
MCP_SERVERS_CONFIG_PATH: /app/config/mcp_servers.docker.yaml
|
||||
depends_on:
|
||||
- telecom-mcp
|
||||
- retail-mcp
|
||||
```
|
||||
|
||||
### How to test MCP tools
|
||||
|
||||
|
||||
### Direct health checks on the servers
|
||||
|
||||
```
|
||||
curl http://localhost:8100/health
|
||||
curl http://localhost:8200/health
|
||||
```
|
||||
|
||||
### List tools directly from Telecom MCP
|
||||
|
||||
```
|
||||
curl http://localhost:8100/mcp/tools/list
|
||||
```
|
||||
|
||||
### Call a tool directly on Telecom MCP
|
||||
|
||||
```
|
||||
curl -X POST http://localhost:8100/mcp/tools/call -H 'Content-Type: application/json' -d '{
|
||||
"tool_name": "consultar_fatura",
|
||||
"arguments": {
|
||||
"msisdn": "11999999999",
|
||||
"invoice_id": "INV-001"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Call a tool directly on Retail MCP
|
||||
|
||||
```
|
||||
curl -X POST http://localhost:8200/mcp/tools/call -H 'Content-Type: application/json' -d '{
|
||||
"tool_name": "consultar_pedido",
|
||||
"arguments": {
|
||||
"order_id": "PED-1001",
|
||||
"customer_id": "C-001"
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### Test through the agent backend
|
||||
|
||||
After starting the MCP servers and backend, the backend provides debug endpoints to list and call tools through `MCPToolRouter`.
|
||||
```
|
||||
cd agent_template_backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e ../agent_framework
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --reload --reload-dir app --reload-dir config --port 8000
|
||||
curl http://localhost:8000/debug/mcp/tools
|
||||
|
||||
curl -X POST http://localhost:8000/debug/mcp/call/consultar_fatura -H 'Content-Type: application/json' -d '{"msisdn":"11999999999","invoice_id":"INV-001"}'
|
||||
```
|
||||
|
||||
### How the agent calls MCP in the flow
|
||||
|
||||
The agent does not need to know the server URL. It calls a logical tool through `MCPToolRouter`. The expected flow is:
|
||||
```
|
||||
Usuário
|
||||
-> FastAPI /gateway/message
|
||||
-> Guardrails de input
|
||||
-> Router ou Supervisor escolhe o agente
|
||||
-> LangGraph executa o agent graph
|
||||
-> Agent decide usar uma tool
|
||||
-> MCPToolRouter.call("consultar_fatura", {...})
|
||||
-> MCPRegistry resolve servidor telecom
|
||||
-> MCPHttpClient chama http://localhost:8100/mcp/tools/call
|
||||
-> Resultado volta ao agent graph
|
||||
-> Guardrails de output
|
||||
-> Judges
|
||||
-> Resposta final
|
||||
```
|
||||
|
||||
### Conceptual Python example
|
||||
|
||||
```
|
||||
result = await tool_router.call(
|
||||
"consultar_fatura",
|
||||
{
|
||||
"msisdn": context.get("msisdn"),
|
||||
"invoice_id": context.get("invoice_id"),
|
||||
},
|
||||
)
|
||||
|
||||
if result.ok:
|
||||
dados_fatura = result.result
|
||||
else:
|
||||
# fallback controlado, telemetria e resposta segura
|
||||
erro = result.error
|
||||
```
|
||||
|
||||
### Example through a gateway message
|
||||
|
||||
```
|
||||
curl -X POST http://localhost:8000/gateway/message -H 'Content-Type: application/json' -d '{
|
||||
"channel": "web",
|
||||
"payload": {
|
||||
"session_id": "sess-tel-1",
|
||||
"message": "Minha fatura veio alta",
|
||||
"context": {
|
||||
"msisdn": "11999999999",
|
||||
"invoice_id": "INV-001"
|
||||
}
|
||||
}
|
||||
}'
|
||||
curl -X POST http://localhost:8000/gateway/message -H 'Content-Type: application/json' -d '{
|
||||
"channel": "web",
|
||||
"payload": {
|
||||
"session_id": "sess-ret-1",
|
||||
"message": "Meu pedido não chegou",
|
||||
"context": {
|
||||
"order_id": "PED-1001",
|
||||
"customer_id": "C-001"
|
||||
}
|
||||
}
|
||||
}'
|
||||
```
|
||||
|
||||
### How to configure new servers and tools
|
||||
|
||||
|
||||
### Add a new MCP Server
|
||||
|
||||
Edit `agent_template_backend/config/mcp_servers.yaml` for local execution:
|
||||
```
|
||||
servers:
|
||||
crm:
|
||||
transport: http
|
||||
endpoint: http://localhost:8300/mcp
|
||||
enabled: true
|
||||
description: MCP Server de CRM.
|
||||
```
|
||||
Edit `agent_template_backend/config/mcp_servers.docker.yaml` for Docker execution:
|
||||
```
|
||||
servers:
|
||||
crm:
|
||||
transport: http
|
||||
endpoint: http://crm-mcp:8300/mcp
|
||||
enabled: true
|
||||
description: MCP Server de CRM via docker-compose.
|
||||
```
|
||||
|
||||
### Register a new tool
|
||||
|
||||
Edit `agent_template_backend/config/tools.yaml`:
|
||||
```
|
||||
tools:
|
||||
consultar_cliente:
|
||||
description: Consulta dados cadastrais resumidos do cliente.
|
||||
mcp_server: crm
|
||||
enabled: true
|
||||
args_schema:
|
||||
customer_id: string
|
||||
document_id: string
|
||||
```
|
||||
|
||||
### Implement the endpoint in the MCP server
|
||||
|
||||
```
|
||||
TOOLS = {
|
||||
"consultar_cliente": {
|
||||
"description": "Consulta dados cadastrais resumidos do cliente.",
|
||||
"input_schema": {
|
||||
"customer_id": "string",
|
||||
"document_id": "string"
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
@app.post("/mcp/tools/call")
|
||||
async def call_tool(call: ToolCall):
|
||||
if call.tool_name == "consultar_cliente":
|
||||
return {
|
||||
"ok": True,
|
||||
"result": {
|
||||
"customer_id": call.arguments.get("customer_id"),
|
||||
"status": "ATIVO",
|
||||
"segmento": "PREMIUM"
|
||||
},
|
||||
"metadata": {"server": "crm", "tool": "consultar_cliente"}
|
||||
}
|
||||
```
|
||||
|
||||
### How to isolate MCP by agent
|
||||
|
||||
In a multi-agent architecture, not every agent should see every tool. The orders agent may use `consultar_pedido` and `consultar_entrega`. The billing agent may use `consultar_fatura` and `consultar_pagamentos`. This isolation reduces operational risk, improves governance, and simplifies each agent's prompt.
|
||||
|
||||
### Simple option: allowlist per agent
|
||||
|
||||
```
|
||||
agents:
|
||||
- agent_id: billing_agent
|
||||
allowed_tools:
|
||||
- consultar_fatura
|
||||
- consultar_pagamentos
|
||||
- consultar_plano
|
||||
- listar_servicos
|
||||
|
||||
- agent_id: orders_agent
|
||||
allowed_tools:
|
||||
- consultar_pedido
|
||||
- consultar_entrega
|
||||
- solicitar_troca
|
||||
- solicitar_devolucao
|
||||
```
|
||||
|
||||
### Recommended option: tools by configuration file
|
||||
|
||||
For large projects, each agent can have its own `tools.yaml`, `guardrails.yaml`, and `judges.yaml`. This maintains real isolation by agent and makes versioning easier.
|
||||
```
|
||||
config/agents/telecom_contas/
|
||||
prompt_policy.yaml
|
||||
guardrails.yaml
|
||||
judges.yaml
|
||||
tools.yaml
|
||||
|
||||
config/agents/retail_orders/
|
||||
prompt_policy.yaml
|
||||
guardrails.yaml
|
||||
judges.yaml
|
||||
tools.yaml
|
||||
```
|
||||
|
||||
### How to deploy with Docker and OCI
|
||||
|
||||
|
||||
### Local deployment with Docker Compose
|
||||
|
||||
The current `docker-compose.yml` already has separate services for `telecom-mcp`, `retail-mcp`, backend, and frontend. This separation is correct because MCP Servers should be independently scalable and versionable from the agent backend.
|
||||
```
|
||||
docker compose up --build
|
||||
|
||||
# URLs externas para teste local:
|
||||
http://localhost:8100/health
|
||||
http://localhost:8200/health
|
||||
http://localhost:8000/debug/mcp/tools
|
||||
http://localhost:5173
|
||||
```
|
||||
|
||||
### Deployment on OCI/OKE
|
||||
|
||||
In Kubernetes/OKE, each MCP Server should be deployed as a Deployment + Service. The agent backend points to the Service's internal DNS. Conceptual example:
|
||||
```
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: telecom-mcp
|
||||
spec:
|
||||
selector:
|
||||
app: telecom-mcp
|
||||
ports:
|
||||
- port: 8100
|
||||
targetPort: 8100
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: telecom-mcp
|
||||
spec:
|
||||
replicas: 2
|
||||
selector:
|
||||
matchLabels:
|
||||
app: telecom-mcp
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: telecom-mcp
|
||||
spec:
|
||||
containers:
|
||||
- name: telecom-mcp
|
||||
image: <registry>/telecom-mcp:1.0.0
|
||||
ports:
|
||||
- containerPort: 8100
|
||||
```
|
||||
|
||||
### Backend configuration in Kubernetes
|
||||
|
||||
```
|
||||
servers:
|
||||
telecom:
|
||||
transport: http
|
||||
endpoint: http://telecom-mcp.default.svc.cluster.local:8100/mcp
|
||||
enabled: true
|
||||
|
||||
retail:
|
||||
transport: http
|
||||
endpoint: http://retail-mcp.default.svc.cluster.local:8200/mcp
|
||||
enabled: true
|
||||
```
|
||||
|
||||
### Security, guardrails, and observability
|
||||
|
||||
MCP greatly increases agent capability, but also increases the attack and operational-risk surface. A tool can query sensitive data, open protocols/cases, cancel services, generate credits, or execute business actions. Therefore, the integration must be protected before, during, and after the call.
|
||||
|
||||
### Minimum security checklist
|
||||
|
||||
- Every tool must have a clear description and argument schema.
|
||||
- Every action tool must require explicit user confirmation before execution.
|
||||
- Each agent must have a tool allowlist.
|
||||
- Sensitive data returned by MCP must pass through masking/sanitization before the final response.
|
||||
- Every MCP call must generate a trace/span/event in Langfuse or OpenTelemetry.
|
||||
- Timeouts and retry limits must be configured per tool or per server.
|
||||
- Do not expose MCP Servers directly to the internet without authentication, TLS, and network controls.
|
||||
- Separate read-only tools from transactional tools.
|
||||
|
||||
### Recommended telemetry
|
||||
|
||||
```
|
||||
span: mcp.tool_call
|
||||
attributes:
|
||||
tenant_id
|
||||
agent_id
|
||||
session_id
|
||||
tool_name
|
||||
mcp_server
|
||||
latency_ms
|
||||
ok
|
||||
error
|
||||
input_argument_keys
|
||||
result_size
|
||||
|
||||
event: mcp.tool_call.completed
|
||||
metadata:
|
||||
tool_name
|
||||
server
|
||||
ok
|
||||
error
|
||||
```
|
||||
|
||||
### Evolution toward official MCP
|
||||
|
||||
The current project uses a simplified HTTP contract. For enterprise production there are two options. The first is to keep this internal contract for simplicity, provided it is well documented, secure, and versioned. The second is to evolve to an official MCP client/server with JSON-RPC, stdio, or Streamable HTTP.
|
||||
|
||||
### Complete developer step-by-step
|
||||
|
||||
```
|
||||
# 1. Baixar e abrir o projeto
|
||||
cd projeto_multi_agent_isolado
|
||||
|
||||
# 2. Subir servidores MCP de exemplo
|
||||
bash ./scripts/run_mcp_servers.sh
|
||||
|
||||
# 3. Em outro terminal, subir backend
|
||||
cd agent_template_backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e ../agent_framework
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --reload --reload-dir app --reload-dir config --port 8000
|
||||
|
||||
# 4. Validar tools carregadas pelo backend
|
||||
curl http://localhost:8000/debug/mcp/tools
|
||||
|
||||
# 5. Chamar tool Telecom
|
||||
curl -X POST http://localhost:8000/debug/mcp/call/consultar_fatura -H 'Content-Type: application/json' -d '{"msisdn":"11999999999","invoice_id":"INV-001"}'
|
||||
|
||||
# 6. Chamar tool Retail
|
||||
curl -X POST http://localhost:8000/debug/mcp/call/consultar_pedido -H 'Content-Type: application/json' -d '{"order_id":"PED-1001","customer_id":"C-001"}'
|
||||
|
||||
# 7. Testar pelo gateway conversacional
|
||||
curl -X POST http://localhost:8000/gateway/message -H 'Content-Type: application/json' -d '{"channel":"web","payload":{"session_id":"sess-ret-1","message":"Meu pedido não chegou","context":{"order_id":"PED-1001","customer_id":"C-001"}}}'
|
||||
```
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
|
||||
### References
|
||||
|
||||
- Model Context Protocol Specification: https://modelcontextprotocol.io/specification
|
||||
- MCP Transports: https://modelcontextprotocol.io/specification/2025-11-25/basic/transports
|
||||
- MCP Resources: https://modelcontextprotocol.io/specification/2025-06-18/server/resources
|
||||
- Reference MCP Servers: https://github.com/modelcontextprotocol/servers
|
||||
- LangChain MCP Adapters: https://docs.langchain.com/oss/python/langchain/mcp
|
||||
- Project files: `agent_framework/src/agent_framework/mcp/*`, `agent_template_backend/config/mcp_servers.yaml`, `agent_template_backend/config/tools.yaml`, `mcp_servers/*`
|
||||
|
||||
### Read-only and transactional policies
|
||||
|
||||
The framework applies a minimal conversational policy immediately before the MCP call. `read_only` classification identifies queries; `transactional` identifies operations that change state. Authorization, idempotency, validation, and atomicity remain the responsibility of the MCP Server.
|
||||
|
||||
### Backend configuration
|
||||
|
||||
The configuration is optional and lives in `config/tool_policies.yaml` in `agent_template_backend`. The path can be set through `TOOL_POLICIES_PATH`. Do not place domain policies inside the shared library.
|
||||
Example:
|
||||
defaults:
|
||||
operation_type: read_only
|
||||
require_confirmation: false
|
||||
tool_policies:
|
||||
alterar_plano:
|
||||
operation_type: transactional
|
||||
require_confirmation: true
|
||||
requires: [new_plan_id]
|
||||
|
||||
### Execution and compatibility
|
||||
|
||||
- Confirmation must arrive as `confirmed: true` or `confirmation: true`; text with value `true` is not sufficient.
|
||||
- If `tool_policies.yaml` does not exist, `tool_type`, `requires`, `confirmation_required`, and `execution_policy` from `tools.yaml` remain valid.
|
||||
- Old tools without policy continue to work without behavior changes.
|
||||
- A blocked call does not reach MCP and returns metadata `blocked_by_policy`, `operation_type`, and `policy_source`.
|
||||
|
||||
### Read-only and transactional policies
|
||||
|
||||
> Content consolidated from `Documentacao/README_TOOL_POLICIES.md`.
|
||||
|
||||
### Goal
|
||||
|
||||
The framework distinguishes query operations (`read_only`) from operations that change state (`transactional`) immediately before the MCP call. This classification does not replace authorization, idempotency, or MCP-server business rules; it only adds minimal conversational protection, especially explicit confirmation.
|
||||
|
||||
### Where to configure
|
||||
|
||||
Configuration belongs to the application backend:
|
||||
|
||||
```text
|
||||
templates/agent_template_backend/config/tool_policies.yaml
|
||||
```
|
||||
|
||||
The shared library contains only the loader and validation. The path is optional:
|
||||
|
||||
```dotenv
|
||||
TOOL_POLICIES_PATH=./config/tool_policies.yaml
|
||||
```
|
||||
|
||||
### Example
|
||||
|
||||
```yaml
|
||||
version: 1
|
||||
|
||||
defaults:
|
||||
operation_type: read_only
|
||||
require_confirmation: false
|
||||
|
||||
tool_policies:
|
||||
consultar_plano:
|
||||
operation_type: read_only
|
||||
|
||||
alterar_plano:
|
||||
operation_type: transactional
|
||||
require_confirmation: true
|
||||
requires: [new_plan_id]
|
||||
```
|
||||
|
||||
To execute `alterar_plano`, the arguments must contain `new_plan_id` and a literal boolean confirmation:
|
||||
|
||||
```json
|
||||
{"new_plan_id": "CONTROLE_100", "confirmed": true}
|
||||
```
|
||||
|
||||
`"confirmation": true` is also accepted. Strings such as `"true"` are not accepted as confirmation.
|
||||
|
||||
### Compatibility
|
||||
|
||||
- If `tool_policies.yaml` does not exist, the framework continues to use `tool_type`, `requires`, `confirmation_required`, and `execution_policy` from `tools.yaml`.
|
||||
- Old tools without policy continue to execute as before.
|
||||
- An explicit policy in the new file takes precedence for that tool's `operation_type` and confirmation.
|
||||
- The `tools.yaml` catalog remains the source for endpoint, schema, enablement, and cache.
|
||||
- The new file must not be placed in `libs/agent_framework`, because decisions vary by application and domain.
|
||||
|
||||
### Execution flow
|
||||
|
||||
```text
|
||||
agente -> MCPToolRouter -> validação da política -> mapeamento de parâmetros -> MCP Gateway/Server
|
||||
```
|
||||
|
||||
A blocked call returns `ok=false`, `metadata.blocked_by_policy=true`, the operation type, and the policy source. The MCP server remains the final authority for authentication, authorization, validation, idempotency, and business transaction.
|
||||
|
||||
### Recommended migration
|
||||
|
||||
1. Update the library without creating the file: legacy behavior remains.
|
||||
2. Create `config/tool_policies.yaml` in the backend.
|
||||
3. Initially register only transactional operations that require confirmation.
|
||||
4. Test calls without confirmation, with boolean confirmation, and with missing required fields.
|
||||
5. Gradually remove duplicate confirmation settings from `tools.yaml` when all consuming templates already use the new configuration.
|
||||
|
||||
|
||||
### Minimum transactional runtime (binding fix)
|
||||
|
||||
The routing `mcp_tools` list is an **allowlist**, not an instruction to execute every tool. The runtime now:
|
||||
|
||||
1. automatically executes only `read_only` tools;
|
||||
2. selects at most one transactional action compatible with the user's request;
|
||||
3. when `require_confirmation: true`, persists `pending_tool_call` and `transaction_status: AWAITING_CONFIRMATION`;
|
||||
4. on the confirmation turn, reuses the same call and executes it with `confirmed: true`;
|
||||
5. publishes `available_mcp_tools`, `selected_tool_call`, `tool_policy_result`, `confirmation_required`, and `confirmation_received` in state.
|
||||
|
||||
For the example scenario, order `123` (or `PED-ENTREGUE`) returns `ENTREGUE` in Retail MCP. Use:
|
||||
|
||||
```text
|
||||
Quero devolver o pedido 123 porque me arrependi da compra.
|
||||
Sim, confirmo a devolução.
|
||||
```
|
||||
|
||||
The MCP contract was standardized to use `reason` in both the catalog and FastMCP server. `tool_policies.yaml` takes precedence over legacy fields in `tools.yaml`; these remain aligned in the templates for compatibility.
|
||||
|
||||
### Tool-policy integration and compatibility
|
||||
|
||||
> Content consolidated from `Documentacao/RELEASE_NOTES_TOOL_POLICIES.md`.
|
||||
|
||||
### Changes
|
||||
|
||||
- New optional `ToolPolicyRegistry` in the shared library.
|
||||
- Central validation in `MCPToolRouter`, including direct calls.
|
||||
- Minimum types `read_only` and `transactional`.
|
||||
- Strict confirmation through `confirmed: true` or `confirmation: true`.
|
||||
- Optional support for required fields per policy.
|
||||
- Automatic fallback to `tool_type`, `requires`, `confirmation_required`, and `execution_policy` from `tools.yaml`.
|
||||
- `config/tool_policies.yaml` and the `TOOL_POLICIES_PATH` variable in the main templates, Day Zero, and `Tuning-Performance/Normal` and `Tuning-Performance/Route_Stickness` variants.
|
||||
- Unit policy and compatibility tests added in `tests/unit/test_tool_policies.py`.
|
||||
|
||||
### Checks performed
|
||||
|
||||
- Compilation of `libs`, `templates`, `Tuning-Performance`, and `tests`: passed.
|
||||
- Structural validation of the six YAML files: passed.
|
||||
- Isolated loader cases (transactional policy, confirmation, missing file, and missing registration): passed.
|
||||
- Rendering of both updated Word manuals: passed, with no clipping or overlap on the added pages.
|
||||
|
||||
### Validation-environment limitation
|
||||
|
||||
The `pytest` suite was prepared but could not be fully executed in this environment because `pytest` and project runtime dependencies were not installed and access to the package index timed out. To reproduce in a project environment:
|
||||
|
||||
```bash
|
||||
PYTHONPATH=libs/agent_framework/src:templates/agent_template_backend python -m pytest -q
|
||||
```
|
||||
|
||||
### Backend/MCP integration correction
|
||||
- `mcp_tools` is now treated as an allowlist.
|
||||
- Actions are no longer automatically executed together with queries.
|
||||
- Transactional confirmation is persisted and resumed on the next turn.
|
||||
- `reason`/`motivo` incompatibility in Retail MCP was corrected.
|
||||
- A deterministic delivered order was added for tests (`123`).
|
||||
- The generic keyword `produto` was removed from the Telecom intent to avoid collisions with Retail returns.
|
||||
- `Normal` and `Route_Stickness` templates in `Tuning-Performance` were synchronized.
|
||||
|
||||
### Contextual MCP parameter extraction
|
||||
|
||||
> Content consolidated from `Documentacao/RELEASE_NOTES_MCP_PARAMETER_EXTRACTION_FIX.md`.
|
||||
|
||||
### Problem fixed
|
||||
|
||||
The `extract` block in `mcp_parameter_mapping.yaml` existed in configuration and documentation, but it was not executed by the runtime. In addition, Business Context values could overwrite explicit arguments, causing `contract_key` to replace the `order_id` provided by the user.
|
||||
|
||||
### Fixes
|
||||
|
||||
- implementation of generic `strategy: llm` extraction after tool selection;
|
||||
- preserved support for `strategy: month_name_pt`;
|
||||
- dedicated `mcp_parameter_extraction` profile;
|
||||
- `llm.mcp_parameter_extraction` telemetry;
|
||||
- `extract` is no longer interpreted as simple mapping;
|
||||
- explicit/extracted arguments take precedence over Business Context;
|
||||
- removal of `contract_key: order_id` from templates;
|
||||
- `order_id` configured as `string`;
|
||||
- update of `Tuning-Performance` variants.
|
||||
|
||||
### Expected result
|
||||
|
||||
For the message `consultar pedido 123`, the MCP call must receive `order_id=123`, even when Business Context contains a different `contract_key`.
|
||||
|
||||
### Local use of MCP tools
|
||||
|
||||
> Content consolidated from `Documentacao/README_MCP.md`.
|
||||
|
||||
This version adds an MCP layer to the framework:
|
||||
|
||||
- `agent_framework.mcp.MCPToolRouter`
|
||||
- `agent_template_backend/config/mcp_servers.yaml`
|
||||
- `agent_template_backend/config/tools.yaml`
|
||||
- `mcp_servers/telecom_mcp_server`
|
||||
- `mcp_servers/retail_mcp_server`
|
||||
|
||||
### Start locally
|
||||
|
||||
Terminal 1:
|
||||
|
||||
```bash
|
||||
bash ./scripts/run_mcp_servers.sh
|
||||
```
|
||||
|
||||
Terminal 2:
|
||||
|
||||
```bash
|
||||
cd agent_template_backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -e ../agent_framework
|
||||
pip install -r requirements.txt
|
||||
uvicorn app.main:app --reload --reload-dir app --reload-dir config --port 8000
|
||||
```
|
||||
|
||||
Terminal 3:
|
||||
|
||||
```bash
|
||||
cd agent_frontend
|
||||
python -m http.server 5173
|
||||
```
|
||||
|
||||
### Quick tests
|
||||
|
||||
List MCP tools loaded by the backend:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/debug/mcp/tools
|
||||
```
|
||||
|
||||
Call a tool directly through the backend:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/debug/mcp/call/consultar_fatura \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"msisdn":"11999999999","invoice_id":"INV-001"}'
|
||||
```
|
||||
|
||||
Telecom routing + MCP:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"session_id":"sess-tel-1","message":"Minha fatura veio alta","context":{"msisdn":"11999999999","invoice_id":"INV-001"}}}'
|
||||
```
|
||||
|
||||
Retail routing + MCP:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"session_id":"sess-ret-1","message":"Meu pedido não chegou","context":{"order_id":"PED-1001","customer_id":"C-001"}}}'
|
||||
```
|
||||
|
||||
### Docker Compose
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
In compose, the backend uses `config/mcp_servers.docker.yaml` to point to `telecom-mcp` and `retail-mcp`.
|
||||
|
||||
### Read-only and transactional operations
|
||||
|
||||
Use `config/tool_policies.yaml` in the backend to classify only operations that need additional handling. Validation is applied in the central router before the MCP Gateway/Server. The file is optional and older templates continue using the policies already present in `tools.yaml`. Full configuration and the migration procedure are in [README_TOOL_POLICIES.md](README_TOOL_POLICIES.md).
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `Documentacao/Manual_Integracao_MCP_Servers_Agent_Framework.docx`
|
||||
- `Documentacao/README_TOOL_POLICIES.md`
|
||||
- `Documentacao/RELEASE_NOTES_TOOL_POLICIES.md`
|
||||
- `Documentacao/RELEASE_NOTES_MCP_PARAMETER_EXTRACTION_FIX.md`
|
||||
- `Documentacao/README_MCP.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,286 @@
|
||||
### Guardrails, Judges, and Transaction Evaluation
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **native/external guardrails, judges, transactional sampling, and grounding**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
Native/external guardrails, judges, transactional sampling, and grounding.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### Guardrails, Judges, and Transaction Evaluation
|
||||
|
||||
Manual for input/output guardrails, agent-specific extensions, external judges, mandatory execution on transactions, and the signals/evidence used during evaluation.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Guardrails implemented in the framework
|
||||
|
||||
> Content consolidated from `Documentacao/README_GUARDRAILS_IMPLEMENTADOS.md`.
|
||||
|
||||
This version adds a pragmatic guardrail layer to `agent_framework`, inspired by separating rails by stage: input, output, retrieval, and execution/tool.
|
||||
|
||||
### Input rails
|
||||
|
||||
- `MSIZE` — blocks excessively large messages.
|
||||
- `MSK` — masks CPF, CNPJ, phone, e-mail, card, postal code, RG, tokens, and keys.
|
||||
- `TOX` — detects toxicity and records severity without blocking by default.
|
||||
- `PINJ` — detects prompt injection and records a score.
|
||||
- `JBRK` — detects jailbreak/bypass roleplay and records a score.
|
||||
- `VLOOP` — blocks repetitive conversational loops.
|
||||
|
||||
### Output rails
|
||||
|
||||
- `PII_OUT` — masks PII in the agent response.
|
||||
- `CMP` — softens absolute promises and excessive guarantee language.
|
||||
- `REVPREC` — blocks verbalization of an operational action without tool confirmation.
|
||||
- `GND` — signals grounding/risk when there is a specific answer without evidence.
|
||||
- `ALUC_RISK` — marks hallucination risk for telemetry and judges.
|
||||
|
||||
### Optional rails
|
||||
|
||||
- `RET_REL` — validates retrieval-chunk relevance using a minimum score.
|
||||
- `TOOL_VAL` — validates MCP/tool name, required arguments, negative values, and allowlist.
|
||||
|
||||
### Contract for authorized protocols in output guardrails
|
||||
|
||||
When a workflow or tool produces a **protocol/reference number that must be shown to the same customer**, the agent integration code must register that value in the output context before output guardrails run:
|
||||
|
||||
```python
|
||||
ctx["expected_protocols"] = [protocol_number]
|
||||
```
|
||||
|
||||
This field is a **framework contract**. It declares that those exact values were produced or validated by the current flow and may therefore be used by output guardrails as authorization evidence.
|
||||
|
||||
Expected flow:
|
||||
|
||||
```text
|
||||
workflow/tool produces protocol
|
||||
↓
|
||||
agent registers it in expected_protocols
|
||||
↓
|
||||
CMP validates that the displayed protocol belongs to the expected values
|
||||
↓
|
||||
DLEX_OUT does not block that protocol merely because it is an identifier
|
||||
↓
|
||||
response may disclose the protocol to the customer
|
||||
```
|
||||
|
||||
Important rules:
|
||||
|
||||
- `expected_protocols` must contain **only protocols actually produced/expected in the current turn or transaction**.
|
||||
- Do not use `expected_protocols` to allow tokens, credentials, arbitrary internal IDs, or third-party data.
|
||||
- Authorization applies only to listed values; any other identifier remains subject to normal `DLEX_OUT` rules.
|
||||
- The value must be propagated **before `output_guardrails`**. Adding it later has no effect.
|
||||
- For transactional responses, keep protocol evidence in the tool/workflow result so `CMP`, `GND`, and observability can correlate the value.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
result = await execute_workflow(...)
|
||||
protocol_number = result.get("protocol_number") or result.get("protocolo_id")
|
||||
if protocol_number:
|
||||
ctx["expected_protocols"] = [str(protocol_number)]
|
||||
```
|
||||
|
||||
#### Troubleshooting: workflow completed but the response was replaced by a safety message
|
||||
|
||||
Typical symptom:
|
||||
|
||||
```text
|
||||
workflow = COMPLETED
|
||||
CMP = allowed
|
||||
DLEX_OUT = blocked because of "internal protocol"
|
||||
final response = "I could not safely validate this response..."
|
||||
```
|
||||
|
||||
Check, in this order:
|
||||
|
||||
1. Is the generated protocol present in the tool/workflow result or evidence?
|
||||
2. Did the agent propagate the same value in `ctx["expected_protocols"]`?
|
||||
3. Was `expected_protocols` populated before `output_guardrails`?
|
||||
4. Is the protocol shown in the response exactly one of the expected values?
|
||||
5. Is `DLEX_OUT` actually blocking another real issue such as a secret, token, or third-party data?
|
||||
|
||||
If `expected_protocols` is absent, the framework must not assume that an arbitrary textual identifier is safe to disclose.
|
||||
|
||||
|
||||
### Files changed
|
||||
|
||||
- `agent_framework/src/agent_framework/guardrails/rails.py`
|
||||
- `agent_framework/src/agent_framework/guardrails/pipeline.py`
|
||||
- `agent_framework/src/agent_framework/guardrails/__init__.py`
|
||||
|
||||
### Quick use
|
||||
|
||||
```python
|
||||
from agent_framework.guardrails.pipeline import GuardrailPipeline
|
||||
|
||||
pipeline = GuardrailPipeline()
|
||||
|
||||
sanitized_input, input_decisions = await pipeline.run_input(
|
||||
user_text,
|
||||
{"history_texts": history_texts},
|
||||
)
|
||||
|
||||
final_answer, output_decisions = await pipeline.run_output(
|
||||
answer,
|
||||
context,
|
||||
)
|
||||
```
|
||||
|
||||
For tools/MCP:
|
||||
|
||||
```python
|
||||
_, decisions = await pipeline.run_tool(
|
||||
"cancelar_produto",
|
||||
{"produto": "VAS", "valor": 0},
|
||||
{
|
||||
"required_args": ["produto"],
|
||||
"allowed_tools": ["cancelar_produto", "consultar_fatura"],
|
||||
},
|
||||
)
|
||||
```
|
||||
|
||||
### SPI for external guardrails and judges
|
||||
|
||||
> Content consolidated from `docs/EXTERNAL_GUARDRAILS_JUDGES.md`.
|
||||
|
||||
`agent_framework_oci` supports agent-owned guardrails and judges without importing domain code into the core.
|
||||
|
||||
```yaml
|
||||
output:
|
||||
- code: ACME_POLICY
|
||||
type: external
|
||||
class: app.extensions.guardrails:AcmePolicyRail
|
||||
```
|
||||
|
||||
```yaml
|
||||
judges:
|
||||
- name: acme_quality
|
||||
type: external
|
||||
class: app.extensions.judges:AcmeQualityJudge
|
||||
threshold: 0.7
|
||||
```
|
||||
|
||||
Native entries remain unchanged. External synchronous `evaluate()` methods execute in worker threads via `asyncio.to_thread`; asynchronous methods execute concurrently on the framework event loop. Judges run concurrently with `asyncio.gather`, preserving YAML result order. Agent plugins should reuse the LLM supplied by the framework rather than instantiate a separate provider.
|
||||
|
||||
The core must not reference a concrete agent package, company, product, telecom identifier, or domain-specific policy. Domain-specific variants belong to the agent and should receive distinct public codes/names.
|
||||
|
||||
### Compatibility rule
|
||||
|
||||
Domain policies must not be replaced by cosmetically generic text inside the core while losing the original policy. The generic core implementation and the agent-specific implementation may coexist; the embedding agent explicitly selects its own code/name in YAML.
|
||||
|
||||
Legacy business validators should migrate to the agent domain. A temporary compatibility shim is acceptable for old imports, but new application code must import the agent-owned implementation.
|
||||
|
||||
### Mandatory judge execution for transactions
|
||||
|
||||
> Content consolidated from `docs/JUDGES_TRANSACTIONAL_SAMPLING_FIX.md`.
|
||||
|
||||
### Problem
|
||||
|
||||
Even with `always_run_for_transactional: true`, judges could be skipped by sampling because the `judge` node sent only `context`, `route`, `intent`, and `mcp_results`. Transactional fields produced by the runtime did not reach `JudgePipeline`.
|
||||
|
||||
### Fix
|
||||
|
||||
The `judge` node now passes:
|
||||
|
||||
- `transaction_status`
|
||||
- `confirmation_required`
|
||||
- `confirmation_received`
|
||||
- `tool_policy_result`
|
||||
- `selected_tool_call`
|
||||
- `pending_tool_call`
|
||||
- `mcp_results` as evidence
|
||||
|
||||
`JudgePipeline` detects transactions through multiple signals and evaluates `always_run_for_transactional` before applying `sample_rate`.
|
||||
|
||||
With the configuration below, common queries continue to be sampled at 25%, but `AWAITING_CONFIRMATION`, `COMPLETED`, `FAILED`, or `CANCELLED` turns always run the judges.
|
||||
|
||||
```yaml
|
||||
enabled: true
|
||||
sample_rate: 0.25
|
||||
always_run_for_transactional: true
|
||||
```
|
||||
|
||||
### Global Supervisor validation
|
||||
|
||||
> Content consolidated from `docs/docs_GLOBAL_SUPERVISOR_VALIDATION.txt`.
|
||||
|
||||
VALIDATION - GLOBAL SUPERVISOR
|
||||
|
||||
Implemented changes:
|
||||
|
||||
1. Framework
|
||||
- agent_framework.global_supervisor.models
|
||||
- agent_framework.global_supervisor.config
|
||||
- agent_framework.global_supervisor.session_store
|
||||
- agent_framework.global_supervisor.router
|
||||
- agent_framework.global_supervisor.client
|
||||
|
||||
2. New service
|
||||
- agent_gateway/app/main.py
|
||||
- agent_gateway/app/settings.py
|
||||
- agent_gateway/config/backends.yaml
|
||||
- agent_gateway/README.md
|
||||
- agent_gateway/Dockerfile
|
||||
- agent_gateway/docs/ARQUITETURA_GLOBAL_SUPERVISOR.md
|
||||
|
||||
3. Docker Compose
|
||||
- agent-gateway service added on port 8010.
|
||||
|
||||
Validations performed:
|
||||
|
||||
- python3 -m compileall -q agent_framework/src/agent_framework/global_supervisor agent_gateway/app
|
||||
Result: OK
|
||||
|
||||
- Hybrid-routing smoke test:
|
||||
Input 1: "My bill is too high" -> billing
|
||||
Input 2: "and this amount?" on the same session_id -> billing via active_backend
|
||||
Result: OK
|
||||
|
||||
- FastAPI app import smoke test:
|
||||
from app.main import app, registry, router
|
||||
Result: OK
|
||||
|
||||
Note:
|
||||
- The gateway SSE proxy was left as a future step. The `/gateway/message/sse` endpoint already routes and forwards as a normal message; for end-to-end SSE, a proxy from `/gateway/events/{session_id}` to the active backend can be implemented.
|
||||
|
||||
### Guardrail event validation
|
||||
|
||||
> Content consolidated from `docs/docs_VALIDATION_GUARDRAILS_IC.txt`.
|
||||
|
||||
VALIDATION REPORT - guardrails parallel fail-fast + observer IC
|
||||
Date: 2026-06-03
|
||||
|
||||
compileall: OK
|
||||
smoke-tests: OK
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `Documentacao/README_GUARDRAILS_IMPLEMENTADOS.md`
|
||||
- `docs/EXTERNAL_GUARDRAILS_JUDGES.md`
|
||||
- `docs/JUDGES_TRANSACTIONAL_SAMPLING_FIX.md`
|
||||
- `docs/docs_GLOBAL_SUPERVISOR_VALIDATION.txt`
|
||||
- `docs/docs_VALIDATION_GUARDRAILS_IC.txt`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,387 @@
|
||||
### RAG, BusinessContext, and Grounding
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **RAG, providers, BusinessContext, retrieved context, and grounding**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
RAG, providers, BusinessContext, retrieved context, and grounding.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### RAG, Enterprise Providers, BusinessContext, and Grounding
|
||||
|
||||
Guide for integrating retrieved knowledge, selecting between RAG providers, configuring KBDB, using samples, MCP sufficiency, and using BusinessContext as a data contract.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Standard RAG Provider versus KBDB Enterprise
|
||||
|
||||
> Content consolidated from `docs/RAG_PROVIDER_KBDB.md`.
|
||||
|
||||
The framework now supports two retrieval backends through the same `RagService` contract, without changing agents or `_retrieve_rag_context()`.
|
||||
|
||||
### Selection
|
||||
|
||||
```env
|
||||
RAG_PROVIDER=standard # default: comportamento anterior
|
||||
# ou
|
||||
RAG_PROVIDER=kbdb # KBDB enterprise
|
||||
```
|
||||
|
||||
Selection is exclusive per process. The two RAG implementations do not run together and do not share vector store, graph store, or ingestion.
|
||||
|
||||
### `standard`
|
||||
|
||||
Fully preserves the existing RAG in `agent_framework_oci`: `VECTOR_STORE_PROVIDER`, `GRAPH_STORE_PROVIDER`, embedding, query rewrite, compression, retrieval guardrails, and generation remain valid.
|
||||
|
||||
### `kbdb`
|
||||
|
||||
The framework integrates only the stable serving port of the KBDB project:
|
||||
|
||||
`PKG_KB_SERVING.SEARCH_KNOWLEDGE_BASE`
|
||||
|
||||
The enterprise pipeline remains external to the agent runtime and preserves its own RAW → SILVER → GOLD architecture, HVI/hybrid search, property graph, publishing, lifecycle, audit, and observability.
|
||||
|
||||
The KBDB envelope is adapted to `RagResult`/`VectorDocument`; therefore existing agents continue calling `_retrieve_rag_context()` and the framework's retrieval guardrails continue after retrieval.
|
||||
|
||||
### Configuration
|
||||
|
||||
```env
|
||||
RAG_PROVIDER=kbdb
|
||||
RAG_TOP_K=5
|
||||
KBDB_DB_USER=KB_USER
|
||||
KBDB_DB_PASSWORD=...
|
||||
KBDB_DB_DSN=...
|
||||
KBDB_DB_WALLET_LOCATION=...
|
||||
KBDB_DB_WALLET_PASSWORD=...
|
||||
KBDB_SEARCH_TYPE=hybrid
|
||||
KBDB_NODE_EXPANSION=true
|
||||
KBDB_NODE_MAX_RELATED=8
|
||||
KBDB_GRAPH_CROSS_REF=false
|
||||
KBDB_MAX_CROSS_REF_HOPS=1
|
||||
KBDB_DOCUMENT_TYPE=customer_safe
|
||||
KBDB_METADATA_JSON=
|
||||
KBDB_MIN_SCORE=
|
||||
```
|
||||
|
||||
When `RAG_PROVIDER=kbdb`, `KBDB_DB_USER`, `KBDB_DB_PASSWORD`, and `KBDB_DB_DSN` are required. KBDB uses an isolated connection because it may reside in another Autonomous database. `KBDB_DB_DSN` follows the same semantics as `ADB_DSN`: use the existing TNS alias in the `tnsnames.ora` from the wallet indicated by `KBDB_DB_WALLET_LOCATION`, not a `tcps://...` URL.
|
||||
|
||||
### Isolation and compatibility
|
||||
|
||||
- `RAG_PROVIDER=standard` does not import or connect to KBDB.
|
||||
- `RAG_PROVIDER=kbdb` does not instantiate the standard RAG vector/graph stores.
|
||||
- Ingestion through `RagService.add_documents()` is not allowed in KBDB mode: it must go through the KBDB pipeline/publishing process.
|
||||
- Query rewrite and context compression remain optional and are applied by the framework's common layer.
|
||||
- `AgentRuntimeMixin._retrieve_rag_context()` and agents remain unchanged.
|
||||
- KBDB failures follow the framework's existing semantics: retrieval is auxiliary evidence and the exception is converted into technical metadata without breaking the user journey.
|
||||
|
||||
|
||||
### Direct tool response and RAG
|
||||
|
||||
The framework no longer considers a structured MCP result, by itself, to be a sufficient user response.
|
||||
|
||||
A `response.renderer` policy defines only **how** to present the result. It does not terminate the flow before RAG/LLM. For a tool to deliberately produce a direct final response, the application must explicitly declare:
|
||||
|
||||
```yaml
|
||||
response:
|
||||
mode: renderer
|
||||
renderer: meu.renderer
|
||||
direct: true
|
||||
```
|
||||
|
||||
Without `direct: true`, the tool result remains MCP evidence and the flow continues to `_retrieve_rag_context()` and LLM composition. This allows, for example, an operational plan query to be combined with KBDB documentary knowledge when the question asks for rules, policies, or explanations.
|
||||
|
||||
The framework core has no fallback by tool name (`consultar_plano`, `consultar_pedido`, etc.). Presentation rules belong to the application/domain.
|
||||
|
||||
|
||||
### MCP sufficiency and grounding
|
||||
|
||||
A successful MCP result **does not** make the framework skip RAG automatically.
|
||||
The domain may declare documentary sufficiency only explicitly in the payload with `rag_sufficient=true` or `knowledge_sufficient=true`. This decision is generic and does not depend on the tool name or telecom/retail keywords.
|
||||
|
||||
For the `kbdb` provider, `KBDB_GROUNDED_ONLY=true` is the default. When KBDB search returns empty, blocked, or error, LLM composition may use facts proven by MCP/business context, but it must not fill the documentary portion using parametric model knowledge. It must state that there is insufficient evidence in the knowledge base.
|
||||
|
||||
ProductAgent events record `IC.PRODUCT_RAG_CONTEXT_EVALUATED` for every attempt/decision and `IC.PRODUCT_RAG_CONTEXT_RETRIEVED` only when context was retrieved. Metadata includes `provider`, `status`, `document_count`, `reason`, `error`, `query`, `namespace`, and `latency_ms`.
|
||||
|
||||
### RAG samples and tests
|
||||
|
||||
> Content consolidated from `docs/README_rag_samples.md`.
|
||||
|
||||
These PDF files are synthetic, searchable sample documents created to validate the RAG embedding and retrieval flow of `agent_template_backend`.
|
||||
|
||||
### Files
|
||||
|
||||
- `01_billing_agent_invoice_policy.pdf` - sample knowledge for `billing_agent`
|
||||
- `02_orders_agent_lifecycle_policy.pdf` - sample knowledge for `orders_agent`
|
||||
- `03_product_agent_catalog_policy.pdf` - sample knowledge for `product_agent`
|
||||
- `04_support_agent_sla_policy.pdf` - sample knowledge for `support_agent`
|
||||
- `05_business_context_rag_flow.pdf` - sample knowledge about BusinessContext, identity.yaml and MCP parameter mapping
|
||||
|
||||
### How to use
|
||||
|
||||
Copy the PDF files to the backend documentation directory:
|
||||
|
||||
```bash
|
||||
mkdir -p agent_template_backend/docs/rag_samples
|
||||
cp *.pdf agent_template_backend/docs/rag_samples/
|
||||
```
|
||||
|
||||
For a local smoke test, use:
|
||||
|
||||
```env
|
||||
VECTOR_STORE_PROVIDER=sqlite
|
||||
EMBEDDING_PROVIDER=mock
|
||||
SQLITE_DB_PATH=./data/agent_framework.db
|
||||
RAG_TOP_K=4
|
||||
```
|
||||
|
||||
Then run:
|
||||
|
||||
```bash
|
||||
python scripts/generate_rag_embeddings.py \
|
||||
--docs-dir ./agent_template_backend/docs/rag_samples \
|
||||
--namespace default
|
||||
```
|
||||
|
||||
For production-like semantic embeddings with OCI Generative AI, use:
|
||||
|
||||
```env
|
||||
VECTOR_STORE_PROVIDER=autonomous
|
||||
EMBEDDING_PROVIDER=oci
|
||||
OCI_COMPARTMENT_ID=ocid1.compartment.oc1..xxxx
|
||||
OCI_REGION=us-chicago-1
|
||||
OCI_EMBEDDING_MODEL=cohere.embed-multilingual-v3.0
|
||||
```
|
||||
|
||||
### Suggested retrieval test questions
|
||||
|
||||
- What is a prorated charge?
|
||||
- When can the OrdersAgent open an exchange request?
|
||||
- Which SKU represents the AI Agents book?
|
||||
- What is the target response for a critical support ticket?
|
||||
- How does BusinessContext map customer_key to MCP tool parameters?
|
||||
|
||||
### BusinessContext v2
|
||||
|
||||
> Content consolidated from `Documentacao/README_TEMPLATE_BUSINESS_CONTEXT_V2.md`.
|
||||
|
||||
This package updates `agent_template_backend` and `agent_frontend` to reflect the new framework, where keys coming from the channel/front end are resolved once into canonical keys and propagated through the layers to the MCP Server.
|
||||
|
||||
### Implemented flow
|
||||
|
||||
1. The front end sends `tenant_id`, `agent_id`, `session_id`, and `business_context`.
|
||||
2. The backend normalizes the message through `ChannelGateway`, preserving the full payload in `context`.
|
||||
3. The backend uses `IdentityResolver` with `config/identity.yaml` to generate `BusinessContext`:
|
||||
- `customer_key`
|
||||
- `contract_key`
|
||||
- `interaction_key`
|
||||
- `account_key`
|
||||
- `resource_key`
|
||||
- `session_key`
|
||||
4. The workflow receives `context.business_context`.
|
||||
5. Example agents no longer build specific arguments such as `msisdn`, `invoice_id`, or `order_id` directly.
|
||||
6. `MCPToolRouter` uses `config/mcp_parameter_mapping.yaml` to convert canonical keys into the actual parameters of each MCP tool.
|
||||
|
||||
### Main files adjusted
|
||||
|
||||
- `agent_template_backend/app/main.py`
|
||||
- loads `IdentityResolver`;
|
||||
- resolves `BusinessContext` per message;
|
||||
- persists keys in session/memory/metadata/SSE;
|
||||
- adds `/debug/identity`.
|
||||
|
||||
- `agent_template_backend/app/agents/runtime.py`
|
||||
- adds centralized `_collect_mcp_context()`;
|
||||
- forwards `business_context` and `original_context` to the MCP Router.
|
||||
|
||||
- `agent_template_backend/app/agents/*_agent.py`
|
||||
- agents now use `_collect_mcp_context()` instead of building specific arguments.
|
||||
|
||||
- `agent_template_backend/config/identity.yaml`
|
||||
- defines how channel/front-end fields feed canonical keys.
|
||||
|
||||
- `agent_template_backend/config/mcp_parameter_mapping.yaml`
|
||||
- defines how canonical keys become real parameters per MCP tool.
|
||||
|
||||
- `agent_frontend/index.html` and `agent_frontend/app.js`
|
||||
- add `tenant`, `agent`, and canonical-key fields;
|
||||
- send `business_context` in the payload;
|
||||
- retain domain aliases for compatibility (`msisdn`, `invoice_id`, `order_id`, etc.).
|
||||
|
||||
### Quick test
|
||||
|
||||
Start backend, frontend, and MCP servers. Then test:
|
||||
|
||||
```bash
|
||||
curl -s http://localhost:8000/health | jq
|
||||
|
||||
curl -s -X POST http://localhost:8000/debug/identity \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"channel":"web",
|
||||
"tenant_id":"default",
|
||||
"agent_id":"telecom_contas",
|
||||
"payload":{
|
||||
"message":"Minha fatura veio alta",
|
||||
"session_id":"teste-001",
|
||||
"msisdn":"11999999999",
|
||||
"invoice_id":"3000131180",
|
||||
"ura_call_id":"URA-123",
|
||||
"business_context":{
|
||||
"customer_key":"11999999999",
|
||||
"contract_key":"3000131180",
|
||||
"interaction_key":"URA-123",
|
||||
"session_key":"teste-001"
|
||||
}
|
||||
}
|
||||
}' | jq
|
||||
|
||||
curl -s -X POST http://localhost:8000/debug/mcp/call/consultar_fatura \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{
|
||||
"business_context": {
|
||||
"customer_key":"11999999999",
|
||||
"contract_key":"3000131180",
|
||||
"interaction_key":"URA-123",
|
||||
"session_key":"teste-001"
|
||||
}
|
||||
}' | jq
|
||||
```
|
||||
|
||||
In the backend log, look for `mcp.tool.mapped`. It should indicate the mapped keys and `has_msisdn=true`, `has_invoice_id=true` for the telecom domain.
|
||||
|
||||
### Operational RAG and cache integration
|
||||
|
||||
> Content consolidated from `Documentacao/README_FIRST_MAX_OPERATIONAL_FIXES.md`.
|
||||
|
||||
This version fixes the gaps identified in the comparison against FIRST.
|
||||
|
||||
### Applied fixes
|
||||
|
||||
### 1. Operational LangGraph checkpoint
|
||||
|
||||
The workflow no longer compiles directly with `MemorySaver()`. The following adapter was created:
|
||||
|
||||
```text
|
||||
agent_framework/checkpoints/langgraph_saver.py
|
||||
```
|
||||
|
||||
It connects LangGraph to the framework's configured repository:
|
||||
|
||||
- `memory`
|
||||
- `sqlite`
|
||||
- `oracle` / `autonomous`
|
||||
|
||||
In the workflow:
|
||||
|
||||
```python
|
||||
builder.compile(checkpointer=create_langgraph_checkpointer(self.settings))
|
||||
```
|
||||
|
||||
### 2. LangGraph telemetry wrapping actual execution
|
||||
|
||||
A node wrapper was added to the workflow:
|
||||
|
||||
```python
|
||||
self._node("billing_agent", self.billing_agent)
|
||||
```
|
||||
|
||||
This way the `langgraph.node.*` span/event wraps actual node execution, not just an empty block.
|
||||
|
||||
Events emitted:
|
||||
|
||||
- `langgraph.node.started`
|
||||
- `langgraph.node.completed`
|
||||
- `langgraph.node.failed`
|
||||
- `langgraph.edge.selected`
|
||||
|
||||
### 3. RAG integrated into agents
|
||||
|
||||
Agents now receive `RagService` and use retrieved context in the prompt:
|
||||
|
||||
- BillingAgent
|
||||
- ProductAgent
|
||||
- OrdersAgent
|
||||
- SupportAgent
|
||||
|
||||
RAG uses:
|
||||
|
||||
- `VECTOR_STORE_PROVIDER=memory|sqlite|oracle|autonomous`
|
||||
- `GRAPH_STORE_PROVIDER=memory|oracle|autonomous`
|
||||
- `RAG_TOP_K`
|
||||
|
||||
### 4. Cache integrated into agent runtime
|
||||
|
||||
The following mixin was created:
|
||||
|
||||
```text
|
||||
agent_template_backend/app/agents/runtime.py
|
||||
```
|
||||
|
||||
It adds:
|
||||
|
||||
- standardized RAG retrieval;
|
||||
- cache key for LLM calls;
|
||||
- hit/miss with telemetry;
|
||||
- distributed cache through `create_cache(settings)`.
|
||||
|
||||
### 5. Unit tests
|
||||
|
||||
The following directory was created:
|
||||
|
||||
```text
|
||||
tests/unit
|
||||
```
|
||||
|
||||
Initial coverage:
|
||||
|
||||
- cache;
|
||||
- SSE;
|
||||
- RAG;
|
||||
- checkpoint saver;
|
||||
- LangGraph telemetry;
|
||||
- agent runtime;
|
||||
- static workflow verification;
|
||||
- main imports.
|
||||
|
||||
Local validation performed:
|
||||
|
||||
```text
|
||||
12 passed
|
||||
```
|
||||
|
||||
### How to test
|
||||
|
||||
```bash
|
||||
cd projeto_agent_framework_first_ready
|
||||
pip install -r agent_template_backend/requirements.txt
|
||||
pytest -q tests/unit
|
||||
```
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `docs/RAG_PROVIDER_KBDB.md`
|
||||
- `docs/README_rag_samples.md`
|
||||
- `Documentacao/README_TEMPLATE_BUSINESS_CONTEXT_V2.md`
|
||||
- `Documentacao/README_FIRST_MAX_OPERATIONAL_FIXES.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,635 @@
|
||||
### Long-Term Memory and Checkpoint
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **LTM, conversation memory, identity-based isolation, and state persistence**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
LTM, conversation memory, identity-based isolation, and state persistence.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### Long-Term Memory and Enterprise Checkpointing
|
||||
|
||||
Implementation manual for durable memory, identity isolation, stores, extraction, LangGraph integration, persistence testing, and the differences among LTM, history, summary, and checkpoint.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Complete Long-Term Memory implementation
|
||||
|
||||
> Content consolidated from `Documentacao/Manual_Long_Term_Memory_PT.md`.
|
||||
|
||||
### Concept
|
||||
|
||||
Long-Term Memory (LTM) is the `agent_framework` capability to store and retrieve durable facts beyond the lifetime of a conversation session.
|
||||
|
||||
Unlike message history, which is normally associated with a `session_id`, long-term memory is associated with the business identity of the user or customer. In the current implementation, this identity is composed of:
|
||||
|
||||
```text
|
||||
tenant_id
|
||||
agent_id
|
||||
customer_key
|
||||
```
|
||||
|
||||
This allows an agent to retrieve preferences, identity information, projects, and constraints even when a new session is created.
|
||||
|
||||
### What it is for
|
||||
|
||||
Long-Term Memory is used to:
|
||||
|
||||
- maintain continuity across sessions;
|
||||
- personalize responses;
|
||||
- avoid making the user repeat information already provided;
|
||||
- reduce the need to send the entire history to the model;
|
||||
- store preferences, current projects, preferred names, and constraints;
|
||||
- isolate memory across tenants, agents, and customers.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
Sessão A:
|
||||
"Me chame de Cris. Minha linguagem preferida é Python."
|
||||
|
||||
Sessão B, com outro session_id e o mesmo customer_key:
|
||||
"O que você lembra sobre mim?"
|
||||
|
||||
Resposta esperada:
|
||||
"Seu nome preferido é Cris e sua linguagem preferida é Python."
|
||||
```
|
||||
|
||||
### Difference among memory types
|
||||
|
||||
### Conversation Memory
|
||||
|
||||
Maintains messages from the current conversation and is normally associated with `session_id`.
|
||||
|
||||
### Summary Memory
|
||||
|
||||
Maintains a conversation summary to reduce the amount of context sent to the model.
|
||||
|
||||
### Long-Term Memory
|
||||
|
||||
Maintains durable facts across sessions and is associated with business identity, especially `customer_key`.
|
||||
|
||||
### Feature components
|
||||
|
||||
### LongTermMemoryManager
|
||||
|
||||
Responsible for coordinating:
|
||||
|
||||
- memory loading;
|
||||
- retrieval by identity;
|
||||
- context rendering;
|
||||
- extraction of new facts;
|
||||
- persistence of facts;
|
||||
- deduplication and updates.
|
||||
|
||||
### LongTermMemoryStore
|
||||
|
||||
Persistence interface used by the manager.
|
||||
|
||||
### SQLiteLongTermMemoryStore
|
||||
|
||||
Reference implementation based on SQLite.
|
||||
|
||||
It is appropriate for:
|
||||
|
||||
- local development;
|
||||
- tests;
|
||||
- demonstrations;
|
||||
- low-scale environments.
|
||||
|
||||
### InMemoryLongTermMemoryStore
|
||||
|
||||
In-memory implementation used for quick tests.
|
||||
|
||||
Its content is lost when the backend process terminates.
|
||||
|
||||
### LongTermMemoryExtractor
|
||||
|
||||
Responsible for identifying durable facts in messages.
|
||||
|
||||
Examples of facts:
|
||||
|
||||
```text
|
||||
preferred_name = Cris
|
||||
preferred_language = Python
|
||||
current_project = Atlas
|
||||
```
|
||||
|
||||
### LongTermMemoryItem
|
||||
|
||||
Model representing a persisted item, including identity, key, value, category, confidence, and metadata.
|
||||
|
||||
### AgentRuntime
|
||||
|
||||
Loads memory before agent execution and injects the context into the prompt.
|
||||
|
||||
### `persist_long_term_memory` node
|
||||
|
||||
LangGraph node responsible for persisting facts after final-response generation and validation.
|
||||
|
||||
### File structure
|
||||
|
||||
```text
|
||||
libs/
|
||||
└── agent_framework/
|
||||
└── src/
|
||||
└── agent_framework/
|
||||
└── memory/
|
||||
├── __init__.py
|
||||
├── long_term_extractor.py
|
||||
├── long_term_memory.py
|
||||
├── long_term_models.py
|
||||
└── long_term_store.py
|
||||
```
|
||||
|
||||
### Execution flow
|
||||
|
||||
```text
|
||||
Mensagem do usuário
|
||||
│
|
||||
▼
|
||||
AgentRuntime.prepare_memory_context()
|
||||
│
|
||||
├── Conversation Memory
|
||||
├── Summary Memory
|
||||
└── Long-Term Memory
|
||||
│
|
||||
▼
|
||||
long_term_memory_context
|
||||
│
|
||||
▼
|
||||
Prompt do agente
|
||||
│
|
||||
▼
|
||||
Agente
|
||||
│
|
||||
▼
|
||||
Guardrails / Judges / Supervisor
|
||||
│
|
||||
▼
|
||||
persist_long_term_memory
|
||||
│
|
||||
▼
|
||||
LongTermMemoryExtractor
|
||||
│
|
||||
▼
|
||||
LongTermMemoryStore
|
||||
```
|
||||
|
||||
### Framework configuration
|
||||
|
||||
### New modules
|
||||
|
||||
Copy the files:
|
||||
|
||||
```text
|
||||
libs/agent_framework/src/agent_framework/memory/long_term_extractor.py
|
||||
libs/agent_framework/src/agent_framework/memory/long_term_memory.py
|
||||
libs/agent_framework/src/agent_framework/memory/long_term_models.py
|
||||
libs/agent_framework/src/agent_framework/memory/long_term_store.py
|
||||
```
|
||||
|
||||
### Updating `memory/__init__.py`
|
||||
|
||||
Export the Long-Term Memory components:
|
||||
|
||||
```python
|
||||
from agent_framework.memory.long_term_memory import (
|
||||
LongTermMemoryManager,
|
||||
create_long_term_memory_manager,
|
||||
)
|
||||
from agent_framework.memory.long_term_models import LongTermMemoryItem
|
||||
from agent_framework.memory.long_term_store import (
|
||||
InMemoryLongTermMemoryStore,
|
||||
LongTermMemoryStore,
|
||||
SQLiteLongTermMemoryStore,
|
||||
create_long_term_memory_store,
|
||||
)
|
||||
```
|
||||
|
||||
### Updating `settings.py`
|
||||
|
||||
Add the configurations:
|
||||
|
||||
```python
|
||||
ENABLE_LONG_TERM_MEMORY: bool = False
|
||||
LONG_TERM_MEMORY_PROVIDER: str = "sqlite"
|
||||
LONG_TERM_MEMORY_SQLITE_PATH: str = "./data/agent_framework.db"
|
||||
LONG_TERM_MEMORY_TABLE: str = "agentfw_long_term_memory"
|
||||
LONG_TERM_MEMORY_MAX_CONTEXT_ITEMS: int = 20
|
||||
LONG_TERM_MEMORY_MIN_CONFIDENCE: float = 0.70
|
||||
LONG_TERM_MEMORY_AUTO_EXTRACT: bool = True
|
||||
LONG_TERM_MEMORY_INJECT_CONTEXT: bool = True
|
||||
```
|
||||
|
||||
### Integration with AgentRuntime
|
||||
|
||||
The runtime must:
|
||||
|
||||
1. check whether the feature is enabled;
|
||||
2. create the manager when necessary;
|
||||
3. retrieve facts by identity;
|
||||
4. populate state;
|
||||
5. inject context into the prompt.
|
||||
|
||||
Fields added to state:
|
||||
|
||||
```python
|
||||
long_term_memories: list[dict]
|
||||
long_term_memory_context: str
|
||||
long_term_memory_write_result: dict
|
||||
```
|
||||
|
||||
### Initialization in AgentWorkflow
|
||||
|
||||
The manager must be created in `AgentWorkflow`:
|
||||
|
||||
```python
|
||||
self.long_term_memory_manager = create_long_term_memory_manager(
|
||||
settings,
|
||||
telemetry=telemetry,
|
||||
)
|
||||
```
|
||||
|
||||
### Correct agent initialization
|
||||
|
||||
`long_term_memory_manager` must not be passed through `agent_kwargs` if the constructors of `BillingAgent`, `ProductAgent`, `OrdersAgent`, and `SupportAgent` do not declare that parameter.
|
||||
|
||||
This initialization causes an error:
|
||||
|
||||
```python
|
||||
agent_kwargs = {
|
||||
"telemetry": telemetry,
|
||||
"settings": settings,
|
||||
"memory": memory,
|
||||
"summary_memory": summary_memory,
|
||||
"long_term_memory_manager": self.long_term_memory_manager,
|
||||
}
|
||||
|
||||
self.billing = BillingAgent(llm, **agent_kwargs)
|
||||
```
|
||||
|
||||
Resulting error:
|
||||
|
||||
```text
|
||||
TypeError: BillingAgent.__init__() got an unexpected keyword argument
|
||||
'long_term_memory_manager'
|
||||
```
|
||||
|
||||
The recommended form is to create agents using the existing signature and inject the manager as an attribute after initialization:
|
||||
|
||||
```python
|
||||
agent_kwargs = {
|
||||
"telemetry": telemetry,
|
||||
"tool_router": getattr(self, "tool_router", None),
|
||||
"rag_service": self.rag_service,
|
||||
"cache": self.cache,
|
||||
"settings": settings,
|
||||
"observer": self.observer,
|
||||
"memory": memory,
|
||||
"summary_memory": summary_memory,
|
||||
}
|
||||
|
||||
self.billing = BillingAgent(llm, **agent_kwargs)
|
||||
self.product = ProductAgent(llm, **agent_kwargs)
|
||||
self.orders = OrdersAgent(llm, **agent_kwargs)
|
||||
self.support = SupportAgent(llm, **agent_kwargs)
|
||||
|
||||
for agent in (
|
||||
self.billing,
|
||||
self.product,
|
||||
self.orders,
|
||||
self.support,
|
||||
):
|
||||
agent.long_term_memory_manager = self.long_term_memory_manager
|
||||
```
|
||||
|
||||
This approach avoids changing every agent constructor and keeps the capability encapsulated in the framework.
|
||||
|
||||
### LangGraph configuration
|
||||
|
||||
Register the node:
|
||||
|
||||
```python
|
||||
builder.add_node(
|
||||
"persist_long_term_memory",
|
||||
self._node(
|
||||
"persist_long_term_memory",
|
||||
self.persist_long_term_memory,
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
Change the flow:
|
||||
|
||||
```python
|
||||
builder.add_edge(
|
||||
"supervisor_review",
|
||||
"persist_long_term_memory",
|
||||
)
|
||||
builder.add_edge(
|
||||
"persist_long_term_memory",
|
||||
"persist",
|
||||
)
|
||||
```
|
||||
|
||||
Implement the method:
|
||||
|
||||
```python
|
||||
async def persist_long_term_memory(
|
||||
self,
|
||||
state: AgentState,
|
||||
) -> dict[str, object]:
|
||||
result = await self.long_term_memory_manager.persist_turn(state)
|
||||
|
||||
return {
|
||||
"long_term_memory_write_result": result,
|
||||
}
|
||||
```
|
||||
|
||||
Final flow:
|
||||
|
||||
```text
|
||||
supervisor_review
|
||||
│
|
||||
▼
|
||||
persist_long_term_memory
|
||||
│
|
||||
▼
|
||||
persist
|
||||
```
|
||||
|
||||
### Environment variables
|
||||
|
||||
```env
|
||||
ENABLE_LONG_TERM_MEMORY=true
|
||||
|
||||
LONG_TERM_MEMORY_PROVIDER=sqlite
|
||||
LONG_TERM_MEMORY_SQLITE_PATH=./data/agent_framework.db
|
||||
LONG_TERM_MEMORY_TABLE=agentfw_long_term_memory
|
||||
|
||||
LONG_TERM_MEMORY_MAX_CONTEXT_ITEMS=20
|
||||
LONG_TERM_MEMORY_MIN_CONFIDENCE=0.70
|
||||
LONG_TERM_MEMORY_AUTO_EXTRACT=true
|
||||
LONG_TERM_MEMORY_INJECT_CONTEXT=true
|
||||
```
|
||||
|
||||
### SQLite database path
|
||||
|
||||
The relative path is resolved from the directory where the backend is started.
|
||||
|
||||
To avoid accidentally creating different databases, prefer an absolute path in development environments:
|
||||
|
||||
```env
|
||||
LONG_TERM_MEMORY_SQLITE_PATH=/mnt/c/Asus_Projects/agent_platform_oci_long_term_memory/data/agent_framework.db
|
||||
```
|
||||
|
||||
Create the directory before starting:
|
||||
|
||||
```bash
|
||||
mkdir -p data
|
||||
```
|
||||
|
||||
### How to test
|
||||
|
||||
### Test 1 — Write
|
||||
|
||||
Send:
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "default:telecom_contas:memory-session-a",
|
||||
"customer_key": "11999999999",
|
||||
"message": "Me chame de Cris. Minha linguagem preferida é Python e meu projeto atual se chama Atlas."
|
||||
}
|
||||
```
|
||||
|
||||
### Test 2 — Retrieval in another session
|
||||
|
||||
Use another `session_id`, keeping the same `customer_key`:
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "default:telecom_contas:memory-session-b",
|
||||
"customer_key": "11999999999",
|
||||
"message": "O que você lembra sobre mim, minhas preferências e meu projeto?"
|
||||
}
|
||||
```
|
||||
|
||||
Expected result:
|
||||
|
||||
```text
|
||||
Seu nome preferido é Cris.
|
||||
Sua linguagem preferida é Python.
|
||||
Seu projeto atual se chama Atlas.
|
||||
```
|
||||
|
||||
### Test 3 — Isolation
|
||||
|
||||
Use another customer:
|
||||
|
||||
```json
|
||||
{
|
||||
"session_id": "default:telecom_contas:memory-session-c",
|
||||
"customer_key": "outro-cliente",
|
||||
"message": "Qual é meu nome preferido e qual é meu projeto atual?"
|
||||
}
|
||||
```
|
||||
|
||||
The data for `11999999999` must not appear.
|
||||
|
||||
### Test 4 — Frontend restart
|
||||
|
||||
Restart or reset the frontend and confirm that it continues sending the same `customer_key`.
|
||||
|
||||
Memory must survive the `session_id` change. Resetting the frontend does not erase SQLite.
|
||||
|
||||
### Test 5 — Backend restart
|
||||
|
||||
Restart Uvicorn and repeat the query.
|
||||
|
||||
With:
|
||||
|
||||
```env
|
||||
LONG_TERM_MEMORY_PROVIDER=sqlite
|
||||
```
|
||||
|
||||
memory must remain available.
|
||||
|
||||
With:
|
||||
|
||||
```env
|
||||
LONG_TERM_MEMORY_PROVIDER=memory
|
||||
```
|
||||
|
||||
memory will be lost when the process terminates.
|
||||
|
||||
### Direct verification in SQLite
|
||||
|
||||
Locate the database:
|
||||
|
||||
```bash
|
||||
find . -name "agent_framework.db" -type f
|
||||
```
|
||||
|
||||
Open it:
|
||||
|
||||
```bash
|
||||
sqlite3 ./data/agent_framework.db
|
||||
```
|
||||
|
||||
Query it:
|
||||
|
||||
```sql
|
||||
SELECT
|
||||
tenant_id,
|
||||
agent_id,
|
||||
customer_key,
|
||||
memory_type,
|
||||
memory_key,
|
||||
memory_value,
|
||||
confidence,
|
||||
created_at,
|
||||
updated_at
|
||||
FROM agentfw_long_term_memory
|
||||
ORDER BY updated_at DESC;
|
||||
```
|
||||
|
||||
### Success criteria
|
||||
|
||||
The implementation is working when:
|
||||
|
||||
- memory is retrieved with another `session_id`;
|
||||
- the same `customer_key` retrieves previous facts;
|
||||
- another `customer_key` cannot access those facts;
|
||||
- restarting the frontend does not erase memory;
|
||||
- restarting the backend does not erase memory when the provider is SQLite;
|
||||
- the `persist_long_term_memory` node executes;
|
||||
- the prompt receives `long_term_memory_context`.
|
||||
|
||||
### Best practices
|
||||
|
||||
- Persist only durable facts.
|
||||
- Do not store the full conversation as Long-Term Memory.
|
||||
- Isolate data by `tenant_id`, `agent_id`, and `customer_key`.
|
||||
- Do not use `session_id` as the user's permanent identity.
|
||||
- Persist only after final validations.
|
||||
- Avoid storing temporary tool results.
|
||||
- Record read, write, update, and failure telemetry.
|
||||
- Define retention and deletion policies.
|
||||
- Use an absolute SQLite path in environments with multiple execution directories.
|
||||
- Migrate to an enterprise database for production and high-availability environments.
|
||||
|
||||
### Reference-implementation limitations
|
||||
|
||||
The current implementation uses rule-based extraction and SQLite as the reference provider.
|
||||
|
||||
Recommended evolutions:
|
||||
|
||||
- fact extraction with LLM;
|
||||
- semantic memory with vectors;
|
||||
- episodic memory;
|
||||
- expiration and versioning;
|
||||
- semantic deduplication;
|
||||
- consent policy;
|
||||
- query and deletion API;
|
||||
- Oracle Autonomous Database provider;
|
||||
- encryption and sensitive-data classification.
|
||||
|
||||
### Enterprise Checkpointing in LangGraph
|
||||
|
||||
> Content consolidated from `Documentacao/README_CHECKPOINT_ENTERPRISE.md`.
|
||||
|
||||
This version adds four capabilities to the LangGraph checkpointer used by the framework:
|
||||
|
||||
1. **Checkpoint Integrity**: each checkpoint is stored inside an envelope containing `schema_version`, `checkpoint_id`, SHA-256 `payload_hash`, and `created_at`. On read, the hash is recalculated. If the payload was truncated, changed, or corrupted, the checkpoint is ignored during recovery.
|
||||
2. **Checkpoint Compaction**: old checkpoints are automatically removed according to `CHECKPOINT_COMPACT_EVERY` and `CHECKPOINT_KEEP_LAST`. This prevents unbounded growth of the `workflow_checkpoints` table.
|
||||
3. **Resilient Checkpointer**: writes and reads use retry with backoff and jitter. The resilient layer works over memory, SQLite, and Oracle/Autonomous Database.
|
||||
4. **Checkpoint Recovery**: when restoring state, the framework scans recent checkpoints and returns the newest valid one, skipping corrupted checkpoints.
|
||||
|
||||
### Configuration
|
||||
|
||||
In `.env`:
|
||||
|
||||
```env
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=sqlite
|
||||
ENABLE_RESILIENT_CHECKPOINTER=true
|
||||
ENABLE_CHECKPOINT_INTEGRITY=true
|
||||
ENABLE_CHECKPOINT_COMPACTION=true
|
||||
CHECKPOINT_COMPACT_EVERY=50
|
||||
CHECKPOINT_KEEP_LAST=20
|
||||
CHECKPOINT_RECOVERY_SCAN_LIMIT=25
|
||||
CHECKPOINT_RETRY_MAX_ATTEMPTS=3
|
||||
CHECKPOINT_RETRY_BASE_DELAY_SECONDS=0.05
|
||||
CHECKPOINT_RETRY_MAX_DELAY_SECONDS=1.0
|
||||
CHECKPOINT_RETRY_JITTER_SECONDS=0.05
|
||||
```
|
||||
|
||||
For production with multiple pods, prefer:
|
||||
|
||||
```env
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=autonomous
|
||||
ADB_USER=...
|
||||
ADB_PASSWORD=...
|
||||
ADB_DSN=...
|
||||
ADB_WALLET_LOCATION=...
|
||||
ADB_TABLE_PREFIX=AGENTFW
|
||||
```
|
||||
|
||||
### Use in LangGraph
|
||||
|
||||
```python
|
||||
from agent_framework.checkpoints import create_langgraph_checkpointer
|
||||
|
||||
checkpointer = create_langgraph_checkpointer(settings)
|
||||
graph = builder.compile(checkpointer=checkpointer)
|
||||
|
||||
config = {"configurable": {"thread_id": session_id}}
|
||||
result = graph.invoke(input_state, config=config)
|
||||
```
|
||||
|
||||
`thread_id` remains the conversation-recovery key. In an environment with a Load Balancer, any pod can resume execution if it uses the same persistent repository.
|
||||
|
||||
### Files changed
|
||||
|
||||
- `agent_framework/src/agent_framework/checkpoints/checkpoint_repository.py`
|
||||
- `agent_framework/src/agent_framework/checkpoints/langgraph_saver.py`
|
||||
- `agent_framework/src/agent_framework/checkpoints/__init__.py`
|
||||
- `agent_framework/src/agent_framework/config/settings.py`
|
||||
- `tests/unit/test_resilient_checkpointer.py`
|
||||
|
||||
### Important note
|
||||
|
||||
The `memory` provider now also uses `RepositoryCheckpointSaver` when `ENABLE_RESILIENT_CHECKPOINTER=true`. To return to LangGraph's pure `MemorySaver` for local tests, configure:
|
||||
|
||||
```env
|
||||
ENABLE_RESILIENT_CHECKPOINTER=false
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=memory
|
||||
```
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `Documentacao/Manual_Long_Term_Memory_PT.md`
|
||||
- `Documentacao/README_CHECKPOINT_ENTERPRISE.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,117 @@
|
||||
### LLM Rich Response and reasoning_content
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **`ainvoke_response()`, inference metadata, and optional `reasoning_content`**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
`ainvoke_response()`, inference metadata, and optional `reasoning_content`.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### LLM Rich Response and reasoning_content
|
||||
|
||||
Guide for using the opt-in structured LLM response API without breaking the legacy `ainvoke()` contract, including `reasoning_content`, usage, model, provider, fallback, and tests.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### Rich LLM response API
|
||||
|
||||
> Content consolidated from `docs/LLM_RICH_RESPONSE.md`.
|
||||
|
||||
### Goal
|
||||
|
||||
The framework keeps `ainvoke()` as the backward-compatible API, returning only `str`, and adds `ainvoke_response()` for consumers that need additional inference metadata, including `reasoning_content` when the model/provider/API makes it available.
|
||||
|
||||
### APIs
|
||||
|
||||
### Legacy API — unchanged
|
||||
|
||||
```python
|
||||
answer = await llm.ainvoke(messages)
|
||||
assert isinstance(answer, str)
|
||||
```
|
||||
|
||||
No existing agent needs to be changed.
|
||||
|
||||
### New rich API — opt-in
|
||||
|
||||
```python
|
||||
response = await llm.ainvoke_response(messages)
|
||||
|
||||
answer = response.content
|
||||
reasoning = response.reasoning_content
|
||||
usage = response.usage
|
||||
model = response.model
|
||||
provider = response.provider
|
||||
```
|
||||
|
||||
`reasoning_content` is `str | None`. `None` is the expected behavior when the model, provider, or API does not expose textual reasoning.
|
||||
|
||||
### Backoffice
|
||||
|
||||
A consumer that previously did:
|
||||
|
||||
```python
|
||||
answer = await llm.ainvoke(messages)
|
||||
template = extract_response(answer)
|
||||
```
|
||||
|
||||
can instead do:
|
||||
|
||||
```python
|
||||
response = await llm.ainvoke_response(messages)
|
||||
template = extract_response(response.content)
|
||||
reasoning_content = response.reasoning_content
|
||||
```
|
||||
|
||||
Logic that expects text continues to receive `response.content`; reasoning remains separate and does not contaminate response, cache, memory, judges, or guardrails.
|
||||
|
||||
### Custom-provider compatibility
|
||||
|
||||
`LLMProvider.ainvoke_response()` has a fallback. An external provider that implements only `ainvoke()` continues to work and automatically receives `LLMResponse(content=<texto>)`, with `reasoning_content=None`.
|
||||
|
||||
Native providers (`mock`, OpenAI-compatible/OCI OpenAI, and OCI SDK) implement the rich response and attempt to preserve reasoning when present.
|
||||
|
||||
### Compatibility guarantees
|
||||
|
||||
- `ainvoke()` continues to return `str`.
|
||||
- No existing router, judge, RAG, memory, cache, or runtime has been migrated to the new API.
|
||||
- `reasoning_content` is never fabricated by the framework.
|
||||
- Missing reasoning does not generate an error.
|
||||
- Existing telemetry output continues to be the final content, without automatically appending reasoning.
|
||||
|
||||
### Tests
|
||||
|
||||
Specific tests are in `tests/unit/test_llm_rich_response.py` and verify:
|
||||
|
||||
1. a legacy provider that implements only `ainvoke()`;
|
||||
2. preservation of the `str` return from `ainvoke()`;
|
||||
3. `LLMResponse` return from `ainvoke_response()`;
|
||||
4. reasoning through a direct attribute;
|
||||
5. reasoning through `model_extra`;
|
||||
6. missing reasoning and extraction in OCI SDK format.
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `docs/LLM_RICH_RESPONSE.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,360 @@
|
||||
### Performance, Cache, and Async Runtime
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **concurrency, cache, reduction of LLM calls, and cross-loop fixes**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
Concurrency, cache, reduction of LLM calls, and cross-loop fixes.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### Performance, Cache, Concurrency, and Async Runtime
|
||||
|
||||
Manual for optimizations on the critical MCP, RAG, and Judge path, reduction of LLM calls, deterministic preemption, and cross-loop deadlock correction in sequencing.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### MCP, RAG, and Judge optimizations
|
||||
|
||||
> Content consolidated from `docs/PERFORMANCE_OPTIMIZATIONS_MCP_JUDGES_RAG.md`.
|
||||
|
||||
- `mcp_tools` remains an allowlist; only the query selected through `selection_keywords` is executed.
|
||||
- `strategy: hybrid` extraction tries a regex `pattern` before the LLM profile.
|
||||
- RAG is skipped when successful MCP evidence is sufficient, except for policy/rule questions.
|
||||
- `mcp_results` is provided as evidence to the groundedness judge.
|
||||
- `judges.yaml` accepts `sample_rate` and `always_run_for_transactional`.
|
||||
- Simple structured queries can return a deterministic response without invoking the agent LLM.
|
||||
|
||||
### Shift from query to transactional action
|
||||
|
||||
Route stickiness is preempted when an explicit keyword configured in `routing.yaml` identifies another intent/agent. Thus, a session in `retail_order_tracking` moves to `retail_support_exchange_return` when it receives requests such as “return order”. In addition, direct responses from read-only tools are blocked when the message contains `selection_keywords` from any registered transactional tool.
|
||||
|
||||
Action words remain in `config/tools.yaml`; the runtime does not maintain hardcoded domain aliases.
|
||||
|
||||
|
||||
### Deterministic preemption for an explicit intent change
|
||||
|
||||
Stickiness does not call a second LLM when the message contains an explicit change that can be recognized deterministically. Multi-token keywords configured in `routing.yaml` accept up to three intermediate tokens while preserving order. Therefore, `cancelar pedido` recognizes `quero cancelar meu pedido`, `cancelar o meu pedido`, and `pode cancelar esse pedido`. In this case the new intent preempts stickiness and the `keyword_match_strategy=ordered_tokens` metadata makes the decision auditable. Messages with no explicit signal continue using route stickiness normally.
|
||||
|
||||
### Cross-loop deadlock fix
|
||||
|
||||
> Content consolidated from `Documentacao/FIX_DEADLOCK_SEQUENCE_CROSS_LOOP.md`.
|
||||
|
||||
### Problem
|
||||
|
||||
The synchronous `agent_framework.observer.event()` API could be called from a worker thread with no active event loop. In that case, the previous implementation ran `asyncio.run(aevent(...))`, creating a temporary new event loop. At the same time, `analytics/tim_sequence.py` shared global `asyncio.Lock` instances (`_mongo_index_lock` and `_memory_lock`) across calls that could come from different event loops.
|
||||
|
||||
On the first Mongo operation, `_ensure_mongo_ttl_index_once()` held `_mongo_index_lock` while creating the TTL index. Contention from another loop could leave the second call waiting indefinitely.
|
||||
|
||||
### Applied changes
|
||||
|
||||
1. `observer.py`
|
||||
- removed `asyncio.run()` from the synchronous `event()` path;
|
||||
- added a dedicated reusable event loop for synchronous calls;
|
||||
- cross-thread submission uses `asyncio.run_coroutine_threadsafe()`;
|
||||
- best-effort loop shutdown when the process terminates.
|
||||
|
||||
2. `analytics/tim_sequence.py`
|
||||
- `_mongo_index_lock`: `asyncio.Lock` -> `threading.Lock`;
|
||||
- `_memory_lock`: `asyncio.Lock` -> `threading.Lock`;
|
||||
- TTL-index initialization moved to a synchronous function protected by a thread lock and called through `asyncio.to_thread()`;
|
||||
- the in-memory fallback counter uses a short thread-safe critical section.
|
||||
|
||||
3. Tests
|
||||
- `tests/test_observer_cross_loop_deadlock_fix.py` validates:
|
||||
- multiple worker threads using `event()` share the same synchronous observer loop;
|
||||
- in-memory sequence remains monotonic across independent event loops;
|
||||
- TTL-index creation happens only once under cross-loop contention.
|
||||
|
||||
### Validation performed
|
||||
|
||||
```bash
|
||||
PYTHONPATH=libs/agent_framework/src pytest -q tests/test_observer_cross_loop_deadlock_fix.py
|
||||
```
|
||||
|
||||
Result: `3 passed`.
|
||||
|
||||
The full repository suite has pre-existing/independent failures unrelated to this change, including collection conflicts for `test_long_term_memory.py`, static template paths, and checkpoint/workflow tests. Those items were not changed by this fix.
|
||||
|
||||
### Operational performance features
|
||||
|
||||
> Content consolidated from `Documentacao/README_MAX_OPERACIONAL.md`.
|
||||
|
||||
This version adds the operational adjustments that were missing to bring the framework closer to the FIRST production standard.
|
||||
|
||||
### Adjustments included in this version
|
||||
|
||||
### 1. Langfuse Enterprise Adapter
|
||||
|
||||
New module:
|
||||
|
||||
```text
|
||||
agent_framework/observability/langfuse_enterprise.py
|
||||
```
|
||||
|
||||
Includes an adapter compatible with Langfuse SDKs v2/v3 for:
|
||||
|
||||
- trace updates;
|
||||
- trace scoring/evaluation;
|
||||
- prompt registry when supported by the SDK;
|
||||
- isolation of Langfuse API differences.
|
||||
|
||||
### 2. Persistent Token and Cost Accounting
|
||||
|
||||
New package:
|
||||
|
||||
```text
|
||||
agent_framework/billing/
|
||||
```
|
||||
|
||||
Includes:
|
||||
|
||||
- `UsageRecord`
|
||||
- `SQLiteUsageRepository`
|
||||
- `OracleUsageRepository`
|
||||
- `create_usage_repository(settings)`
|
||||
|
||||
The LLM provider now records automatically:
|
||||
|
||||
- `prompt_tokens`
|
||||
- `completion_tokens`
|
||||
- `cached_tokens`
|
||||
- `total_tokens`
|
||||
- `cost_usd`
|
||||
- `cost_brl`
|
||||
- `tenant_id`
|
||||
- `agent_id`
|
||||
- `session_id`
|
||||
- `message_id`
|
||||
|
||||
New endpoint:
|
||||
|
||||
```http
|
||||
GET /debug/usage
|
||||
GET /debug/usage?tenant_id=default
|
||||
GET /debug/usage?session_id=<id>
|
||||
```
|
||||
|
||||
### 3. Operational RAG Service
|
||||
|
||||
New module:
|
||||
|
||||
```text
|
||||
agent_framework/rag/rag_service.py
|
||||
```
|
||||
|
||||
Includes:
|
||||
|
||||
- `RagService.add_documents()`
|
||||
- `RagService.retrieve()`
|
||||
- `RagResult.as_prompt_context()`
|
||||
- telemetry for latency, document count, top scores, and graph.
|
||||
|
||||
### 4. New configuration
|
||||
|
||||
Variable added:
|
||||
|
||||
```env
|
||||
USAGE_REPOSITORY_PROVIDER=sqlite
|
||||
```
|
||||
|
||||
Values:
|
||||
|
||||
```text
|
||||
sqlite
|
||||
oracle
|
||||
autonomous
|
||||
```
|
||||
|
||||
### 5. Local operational compatibility
|
||||
|
||||
By default, usage accounting uses SQLite even when everything else is in memory. This makes local testing possible without Oracle.
|
||||
|
||||
### Quick test
|
||||
|
||||
```bash
|
||||
cd agent_template_backend
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
Test a message:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"text":"teste","user_id":"u1","session_id":"s1"}}'
|
||||
```
|
||||
|
||||
Check usage/cost:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/debug/usage
|
||||
```
|
||||
|
||||
### To run closer to a production pattern
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=sqlite
|
||||
MEMORY_REPOSITORY_PROVIDER=sqlite
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=sqlite
|
||||
USAGE_REPOSITORY_PROVIDER=sqlite
|
||||
CACHE_BACKEND_PROVIDER=sqlite
|
||||
VECTOR_STORE_PROVIDER=sqlite
|
||||
ENABLE_LANGFUSE=true
|
||||
LANGFUSE_HOST=http://localhost:3000
|
||||
LANGFUSE_PUBLIC_KEY=...
|
||||
LANGFUSE_SECRET_KEY=...
|
||||
```
|
||||
|
||||
For Autonomous Database:
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=oracle
|
||||
MEMORY_REPOSITORY_PROVIDER=oracle
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=oracle
|
||||
USAGE_REPOSITORY_PROVIDER=oracle
|
||||
CACHE_BACKEND_PROVIDER=oracle
|
||||
VECTOR_STORE_PROVIDER=oracle
|
||||
GRAPH_STORE_PROVIDER=oracle
|
||||
ADB_USER=...
|
||||
ADB_PASSWORD=...
|
||||
ADB_DSN=...
|
||||
ADB_WALLET_LOCATION=...
|
||||
ADB_TABLE_PREFIX=AGENTFW
|
||||
```
|
||||
|
||||
### Final cache, RAG, and telemetry adjustments
|
||||
|
||||
> Content consolidated from `Documentacao/README_FIRST_MAX_OPERATIONAL_FIXES.md`.
|
||||
|
||||
This version fixes the gaps identified in the comparison against FIRST.
|
||||
|
||||
### Applied fixes
|
||||
|
||||
### 1. Operational LangGraph checkpoint
|
||||
|
||||
The workflow no longer compiles directly with `MemorySaver()`. The following adapter was created:
|
||||
|
||||
```text
|
||||
agent_framework/checkpoints/langgraph_saver.py
|
||||
```
|
||||
|
||||
It connects LangGraph to the framework's configured repository:
|
||||
|
||||
- `memory`
|
||||
- `sqlite`
|
||||
- `oracle` / `autonomous`
|
||||
|
||||
In the workflow:
|
||||
|
||||
```python
|
||||
builder.compile(checkpointer=create_langgraph_checkpointer(self.settings))
|
||||
```
|
||||
|
||||
### 2. LangGraph telemetry wrapping actual execution
|
||||
|
||||
A node wrapper was added to the workflow:
|
||||
|
||||
```python
|
||||
self._node("billing_agent", self.billing_agent)
|
||||
```
|
||||
|
||||
This way the `langgraph.node.*` span/event wraps actual node execution, not just an empty block.
|
||||
|
||||
Events emitted:
|
||||
|
||||
- `langgraph.node.started`
|
||||
- `langgraph.node.completed`
|
||||
- `langgraph.node.failed`
|
||||
- `langgraph.edge.selected`
|
||||
|
||||
### 3. RAG integrated into agents
|
||||
|
||||
Agents now receive `RagService` and use retrieved context in the prompt:
|
||||
|
||||
- BillingAgent
|
||||
- ProductAgent
|
||||
- OrdersAgent
|
||||
- SupportAgent
|
||||
|
||||
RAG uses:
|
||||
|
||||
- `VECTOR_STORE_PROVIDER=memory|sqlite|oracle|autonomous`
|
||||
- `GRAPH_STORE_PROVIDER=memory|oracle|autonomous`
|
||||
- `RAG_TOP_K`
|
||||
|
||||
### 4. Cache integrated into agent runtime
|
||||
|
||||
The following mixin was created:
|
||||
|
||||
```text
|
||||
agent_template_backend/app/agents/runtime.py
|
||||
```
|
||||
|
||||
It adds:
|
||||
|
||||
- standardized RAG retrieval;
|
||||
- cache key for LLM calls;
|
||||
- hit/miss with telemetry;
|
||||
- distributed cache through `create_cache(settings)`.
|
||||
|
||||
### 5. Unit tests
|
||||
|
||||
The following directory was created:
|
||||
|
||||
```text
|
||||
tests/unit
|
||||
```
|
||||
|
||||
Initial coverage:
|
||||
|
||||
- cache;
|
||||
- SSE;
|
||||
- RAG;
|
||||
- checkpoint saver;
|
||||
- LangGraph telemetry;
|
||||
- agent runtime;
|
||||
- static workflow verification;
|
||||
- main imports.
|
||||
|
||||
Local validation performed:
|
||||
|
||||
```text
|
||||
12 passed
|
||||
```
|
||||
|
||||
### How to test
|
||||
|
||||
```bash
|
||||
cd projeto_agent_framework_first_ready
|
||||
pip install -r agent_template_backend/requirements.txt
|
||||
pytest -q tests/unit
|
||||
```
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `docs/PERFORMANCE_OPTIMIZATIONS_MCP_JUDGES_RAG.md`
|
||||
- `Documentacao/FIX_DEADLOCK_SEQUENCE_CROSS_LOOP.md`
|
||||
- `Documentacao/README_MAX_OPERACIONAL.md`
|
||||
- `Documentacao/README_FIRST_MAX_OPERATIONAL_FIXES.md`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,689 @@
|
||||
### Observability, Persistence, and Operational Readiness
|
||||
|
||||
### How to use this manual
|
||||
|
||||
This is a **specialized reference manual**. It does not replace the main tutorial.
|
||||
|
||||
- To create an agent from start to finish, use [`README_en.md`](../../../README_en.md).
|
||||
- Use this document when you need to implement, deepen, or diagnose **telemetry, IC/NOC/GRL, correlation, sequencing, persistence, and operational diagnostics**.
|
||||
- Historical examples consolidated here should be read in light of the framework's current API.
|
||||
- In case of divergence, the code for the version and the current `README_en.md` take precedence.
|
||||
|
||||
### Relationship with the main tutorial
|
||||
|
||||
The `README_en.md` presents this capability in the normal development flow. This manual brings together details that were distributed across `docs/`, `Documentacao/`, release notes, validations, and specialized guides.
|
||||
|
||||
The goal here is to answer **“how does this feature work in depth and how do I solve problems with it?”**, without turning this file into a second copy of the main tutorial.
|
||||
|
||||
### Scope
|
||||
|
||||
Telemetry, IC/NOC/GRL, correlation, sequencing, persistence, and operational diagnostics.
|
||||
|
||||
### Consolidated technical content
|
||||
|
||||
### Observability, Persistence, and Operational Readiness
|
||||
|
||||
Consolidated guide to FIRST-ready capabilities: end-to-end correlation, Langfuse, OpenTelemetry, observable SSE, Oracle persistence, token/cost accounting, cache, and LangGraph telemetry.
|
||||
|
||||
### How to use this document
|
||||
|
||||
This is the consolidated development document for this subject. It brings together architecture, configuration, examples, runtime behavior, compatibility, tests, and troubleshooting that were previously distributed across several files. Source sections were preserved when they provided distinct technical details; release notes were incorporated as current behavior or correction history.
|
||||
|
||||
### FIRST-ready foundation and observability
|
||||
|
||||
> Content consolidated from `Documentacao/README_FIRST_READY.md`.
|
||||
|
||||
This version preserves the `meu_projeto_agent_framework` architecture and adds the operational patterns found in the FIRST project.
|
||||
|
||||
### Added features
|
||||
|
||||
1. **SSE following the FIRST pattern**
|
||||
- `GET /gateway/events/{session_id}` for a `text/event-stream`.
|
||||
- `POST /gateway/message/sse` to process a message while emitting SSE events.
|
||||
- Events: `connected`, `flow.start`, `session.upserted`, `message.received`, `workflow.started`, `workflow.completed`, `message.responded`, `flow.end`.
|
||||
- Keepalive configurable through `SSE_KEEPALIVE_SECONDS`.
|
||||
- Per-session lock to prevent concurrency within the same conversation.
|
||||
- Event replay through `Last-Event-ID` or the `last_event_id` query parameter.
|
||||
|
||||
2. **Session and message persistence**
|
||||
- Implemented `sqlite` provider, runnable locally.
|
||||
- `SESSION_REPOSITORY_PROVIDER=sqlite`.
|
||||
- `MEMORY_REPOSITORY_PROVIDER=sqlite`.
|
||||
- Local tables: `agent_sessions`, `agent_messages`.
|
||||
- Idempotency by `message_id`.
|
||||
|
||||
3. **Persistent checkpoint**
|
||||
- Implemented `sqlite` provider for the workflow's final checkpoint.
|
||||
- `CHECKPOINT_REPOSITORY_PROVIDER=sqlite`.
|
||||
- Read endpoint: `GET /sessions/{session_id}/checkpoint`.
|
||||
|
||||
4. **Message history**
|
||||
- Endpoint: `GET /sessions/{session_id}/messages`.
|
||||
- History is used as conversational memory before invoking LangGraph.
|
||||
|
||||
5. **Cache**
|
||||
- New module `agent_framework.cache.cache`.
|
||||
- Supports local in-memory cache and Redis when `ENABLE_REDIS_CACHE=true`.
|
||||
|
||||
6. **RAG / Vector Store**
|
||||
- `agent_framework.rag.vector_store` now includes `InMemoryVectorStore`, `SQLiteVectorStore`, and the `AutonomousVectorStore` contract.
|
||||
- The SQLite version uses local lexical search for development.
|
||||
- The contract allows replacement by Oracle Vector Search without changing the application layer.
|
||||
|
||||
7. **Observability**
|
||||
- Preserves existing Langfuse integration.
|
||||
- Adds gateway/SSE/workflow events with `session_id`, `agent_id`, `tenant_id`, `message_id`, route, and intent.
|
||||
|
||||
### Resulting architecture
|
||||
|
||||
```text
|
||||
Browser
|
||||
|-- POST /gateway/message/sse
|
||||
|-- GET /gateway/events/{session_id}
|
||||
|
|
||||
FastAPI Template Backend
|
||||
|
|
||||
ChannelGateway
|
||||
|
|
||||
SessionRepository + MessageHistory + CheckpointRepository
|
||||
|
|
||||
LangGraph AgentWorkflow
|
||||
|
|
||||
Guardrails -> Router/Supervisor -> Agent -> Output Guardrails -> Judges
|
||||
|
|
||||
Telemetry / Langfuse / OCI Streaming
|
||||
```
|
||||
|
||||
### How to run locally
|
||||
|
||||
```bash
|
||||
cd agent_template_backend
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
pip install -e ../agent_framework
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
Frontend:
|
||||
|
||||
```bash
|
||||
cd agent_frontend
|
||||
python -m http.server 3000
|
||||
```
|
||||
|
||||
Open:
|
||||
|
||||
```text
|
||||
http://localhost:3000
|
||||
```
|
||||
|
||||
### Main variables
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=sqlite
|
||||
MEMORY_REPOSITORY_PROVIDER=sqlite
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=sqlite
|
||||
VECTOR_STORE_PROVIDER=sqlite
|
||||
SQLITE_DB_PATH=./data/agent_framework.db
|
||||
ENABLE_SSE=true
|
||||
SSE_KEEPALIVE_SECONDS=15
|
||||
ENABLE_MESSAGE_IDEMPOTENCY=true
|
||||
```
|
||||
|
||||
### Test with curl
|
||||
|
||||
Normal message:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"text":"teste","message":"teste","session_id":"s1","user_id":"u1","message_id":"m1"}}'
|
||||
```
|
||||
|
||||
Message with SSE:
|
||||
|
||||
```bash
|
||||
curl -N http://localhost:8000/gateway/events/s1
|
||||
```
|
||||
|
||||
In another terminal:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message/sse \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"text":"teste","message":"teste","session_id":"s1","user_id":"u1","message_id":"m2"}}'
|
||||
```
|
||||
|
||||
History:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/sessions/s1/messages
|
||||
```
|
||||
|
||||
Checkpoint:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/sessions/s1/checkpoint
|
||||
```
|
||||
|
||||
### Important note
|
||||
|
||||
The added version is locally executable with SQLite. The `AutonomousSessionRepository`, `DatabaseMessageHistory`, `AutonomousCheckpointRepository`, and `AutonomousVectorStore` classes preserve the Oracle Autonomous Database contract, but in this delivery they use SQLite as the local backend so the project can run and be tested without Oracle infrastructure.
|
||||
|
||||
### FIRST-style Observability evolution
|
||||
|
||||
This version adds an enterprise observability layer to the framework while keeping reusable components inside `agent_framework`.
|
||||
|
||||
### Added components
|
||||
|
||||
```text
|
||||
agent_framework/observability/
|
||||
├── context.py # ContextVar: request_id, session_id, user_id, tenant_id, agent_id, channel, ura_call_id, workflow_id, message_id
|
||||
├── telemetry.py # Facade central: span, event, generation, rag_event, cache_event, checkpoint_event
|
||||
├── event_bus.py # Event bus interno para plugar logs, SSE, OCI Streaming, Elastic, Phoenix etc.
|
||||
├── otel.py # OpenTelemetry opcional via OTLP
|
||||
├── workflow_events.py # workflow.started, node.started, node.completed, edge.selected, workflow.failed
|
||||
├── guardrail_events.py # guardrail.<CODE>.evaluated e guardrail.<CODE>.blocked
|
||||
├── judge_events.py # judge.<NAME>.evaluated
|
||||
├── streaming_events.py # sse.connected, sse.keepalive, sse.event.emitted
|
||||
└── decorators.py # decorator @traced para classes do framework
|
||||
```
|
||||
|
||||
### End-to-end correlation
|
||||
|
||||
Each HTTP call creates or propagates `x-request-id`, and the message flow links:
|
||||
|
||||
```text
|
||||
request_id → tenant_id → agent_id → session_id → user_id → channel → message_id → workflow_id
|
||||
```
|
||||
|
||||
The context uses `ContextVar`, so it works across async calls, FastAPI, LangGraph, and LLM providers.
|
||||
|
||||
### Langfuse
|
||||
|
||||
Enable in `.env`:
|
||||
|
||||
```env
|
||||
ENABLE_LANGFUSE=true
|
||||
LANGFUSE_PUBLIC_KEY=pk-lf-...
|
||||
LANGFUSE_SECRET_KEY=sk-lf-...
|
||||
LANGFUSE_HOST=http://localhost:3000
|
||||
```
|
||||
|
||||
The framework records:
|
||||
|
||||
```text
|
||||
Trace de conversa
|
||||
├── http.request
|
||||
├── agent.gateway_message
|
||||
├── workflow.langgraph.ainvoke
|
||||
├── workflow.input_guardrails
|
||||
│ └── guardrail.<CODE>.evaluated / blocked
|
||||
├── workflow.routing_decision
|
||||
├── workflow.agent.<agent>
|
||||
│ └── generation.<model>
|
||||
├── workflow.output_guardrails
|
||||
├── workflow.judge
|
||||
│ └── judge.<NAME>.evaluated
|
||||
├── workflow.supervisor_review
|
||||
├── workflow.persist
|
||||
└── sse.event.emitted / sse.keepalive
|
||||
```
|
||||
|
||||
### OpenTelemetry
|
||||
|
||||
Enable in `.env`:
|
||||
|
||||
```env
|
||||
ENABLE_OTEL=true
|
||||
OTEL_SERVICE_NAME=agent-framework-template
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318/v1/traces
|
||||
```
|
||||
|
||||
With this configuration, the same spans are exported through OTLP to Elastic, Grafana Tempo, Jaeger, Collector, or another compatible backend.
|
||||
|
||||
### Observable SSE
|
||||
|
||||
`SSEHub` now records events for:
|
||||
|
||||
- opened connection;
|
||||
- event replay;
|
||||
- emitted event;
|
||||
- keepalive;
|
||||
- per-session lock during message processing.
|
||||
|
||||
### Guardrails and Judges
|
||||
|
||||
In addition to aggregate events (`guardrails.input.completed`, `judges.completed`), each individual decision generates its own telemetry:
|
||||
|
||||
```text
|
||||
guardrail.MSK.evaluated
|
||||
guardrail.OOS.blocked
|
||||
judge.response_quality.evaluated
|
||||
judge.groundedness.evaluated
|
||||
```
|
||||
|
||||
### Extension to other backends
|
||||
|
||||
The `Telemetry.event_bus` class allows new handlers to be plugged in without changing the workflow. Example:
|
||||
|
||||
```python
|
||||
async def enviar_para_elastic(event):
|
||||
...
|
||||
|
||||
telemetry.event_bus.subscribe(enviar_para_elastic)
|
||||
```
|
||||
|
||||
|
||||
---
|
||||
|
||||
### Complete FIRST Enterprise evolution
|
||||
|
||||
This version received the components that were missing to bring the framework closer to the operational standard of the FIRST project:
|
||||
|
||||
### Oracle Autonomous Database persistence
|
||||
|
||||
Real Oracle providers were added:
|
||||
|
||||
- `OracleSessionRepository`
|
||||
- `OracleMessageHistory`
|
||||
- `OracleCheckpointRepository`
|
||||
- `OracleCache`
|
||||
- `OracleVectorStore`
|
||||
- `OracleGraphStore`
|
||||
- `OracleStore`
|
||||
|
||||
Tables are created automatically with configurable `ADB_TABLE_PREFIX`:
|
||||
|
||||
- `<PREFIX>_AGENT_SESSION`
|
||||
- `<PREFIX>_AGENT_MESSAGE`
|
||||
- `<PREFIX>_WORKFLOW_CHECKPOINT`
|
||||
- `<PREFIX>_WORKFLOW_CHECKPOINT_WRITE`
|
||||
- `<PREFIX>_WORKFLOW_CHECKPOINT_BLOB`
|
||||
- `<PREFIX>_SSE_EVENT`
|
||||
- `<PREFIX>_CACHE_ENTRY`
|
||||
- `<PREFIX>_RAG_DOCUMENT`
|
||||
- `<PREFIX>_GRAPH_EDGE`
|
||||
|
||||
### Oracle configuration
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=oracle
|
||||
MEMORY_REPOSITORY_PROVIDER=oracle
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=oracle
|
||||
CACHE_BACKEND_PROVIDER=oracle
|
||||
VECTOR_STORE_PROVIDER=oracle
|
||||
GRAPH_STORE_PROVIDER=oracle
|
||||
SSE_STORE_PROVIDER=oracle
|
||||
|
||||
ADB_USER=ADMIN
|
||||
ADB_PASSWORD=***
|
||||
ADB_DSN=meu_adb_high
|
||||
ADB_WALLET_LOCATION=/path/wallet
|
||||
ADB_WALLET_PASSWORD=***
|
||||
ADB_TABLE_PREFIX=AGENTFW
|
||||
```
|
||||
|
||||
### Enterprise SSE
|
||||
|
||||
SSE now includes:
|
||||
|
||||
- per-session lock (`SessionLockManager`)
|
||||
- configurable keepalive
|
||||
- replay through `Last-Event-ID`
|
||||
- event persistence in SQLite or Oracle
|
||||
- connection, replay, keepalive, and disconnection telemetry
|
||||
|
||||
Endpoint:
|
||||
|
||||
```text
|
||||
GET /gateway/events/{session_id}?last_event_id=123
|
||||
```
|
||||
|
||||
### LangGraph Deep Telemetry
|
||||
|
||||
`LangGraphDeepTelemetry` was added with events:
|
||||
|
||||
- `langgraph.node.started`
|
||||
- `langgraph.node.completed`
|
||||
- `langgraph.node.failed`
|
||||
- `langgraph.edge.selected`
|
||||
|
||||
These events are sent to Event Bus, Langfuse, and OpenTelemetry when enabled.
|
||||
|
||||
### Token and Cost Accounting
|
||||
|
||||
The following were added:
|
||||
|
||||
- `TokenUsageCollector`
|
||||
- `CostTracker`
|
||||
- calculation of `prompt_tokens`, `completion_tokens`, `cached_tokens`, `total_tokens`
|
||||
- calculation of `cost_usd` and `cost_brl`
|
||||
|
||||
Optional configuration:
|
||||
|
||||
```env
|
||||
USD_BRL_RATE=5.0
|
||||
MODEL_PRICES_JSON={"openai.gpt-4.1":{"input_per_1m":"2.00","output_per_1m":"8.00"}}
|
||||
```
|
||||
|
||||
### Enterprise Cache
|
||||
|
||||
Cache is now layered:
|
||||
|
||||
```text
|
||||
L1: InMemory
|
||||
L2: Redis, SQLite ou Oracle
|
||||
```
|
||||
|
||||
Configuration:
|
||||
|
||||
```env
|
||||
ENABLE_REDIS_CACHE=true
|
||||
REDIS_URL=redis://localhost:6379/0
|
||||
```
|
||||
|
||||
or:
|
||||
|
||||
```env
|
||||
CACHE_BACKEND_PROVIDER=oracle
|
||||
```
|
||||
|
||||
### Oracle 23ai RAG
|
||||
|
||||
`OracleVectorStore` was added, with support for a `VECTOR` column and `VECTOR_DISTANCE()` when an embedding provider is connected.
|
||||
Without an embedding provider, it keeps a lexical fallback for local development.
|
||||
|
||||
`OracleGraphStore` was also added with an edge table, ready to evolve to PGQL/Property Graph.
|
||||
|
||||
### Langfuse
|
||||
|
||||
Each LLM call now generates a `generation` with:
|
||||
|
||||
- input
|
||||
- output
|
||||
- model
|
||||
- provider
|
||||
- token usage
|
||||
- cost metadata
|
||||
|
||||
In addition, workflow, guardrail, judge, RAG, cache, checkpoint, SSE, and LangGraph spans are published through the same Event Bus.
|
||||
|
||||
### Enterprise Plus extensions
|
||||
|
||||
> Content consolidated from `Documentacao/README_FIRST_ENTERPRISE_PLUS.md`.
|
||||
|
||||
This version evolves the framework in the four requested areas:
|
||||
|
||||
1. **Complete Langfuse Enterprise**
|
||||
- `Telemetry.span()` with trace/session/user/metadata/tags.
|
||||
- `Telemetry.generation()` with `usage`, token/cost metadata, and Langfuse v2/v3 compatibility.
|
||||
- `Telemetry.score()` for judges/evaluations.
|
||||
- Arbitrary events are recorded as safe spans to avoid `Unknown observation type` in Langfuse.
|
||||
|
||||
2. **Complete Token/Cost Accounting**
|
||||
- `TokenUsageCollector` supports `prompt_tokens`, `completion_tokens`, `cached_tokens`, `reasoning_tokens`, and `total_tokens`.
|
||||
- Per-model pricing table through `MODEL_PRICES_JSON`.
|
||||
- USD→BRL conversion through `USD_BRL_RATE`.
|
||||
- Persistence in `UsageRepository` and `/debug/usage` endpoint.
|
||||
|
||||
3. **Distributed Redis**
|
||||
- `DistributedCache`: L1 memory + L2 Redis/SQLite/Oracle.
|
||||
- `RedisCache` with `redis.asyncio` when available and sync fallback.
|
||||
- Namespace through `CACHE_KEY_PREFIX`.
|
||||
- Cache hit/miss/set/delete telemetry.
|
||||
|
||||
4. **Real Oracle Vector + PGQL**
|
||||
- `OracleVectorStore` uses `VECTOR_DISTANCE(..., COSINE)` and `TO_VECTOR()` in Oracle 23ai.
|
||||
- Automatically attempts to create a vector index when supported.
|
||||
- `OracleGraphStore` uses `GRAPH_NODE` and `GRAPH_EDGE` tables.
|
||||
- Supports Property Graph creation and `GRAPH_TABLE`/PGQL queries, with SQL fallback.
|
||||
|
||||
The SSE duplication problem caused by replay + live queue was also fixed using `max_replayed_id` control in `SSEHub.subscribe()`.
|
||||
|
||||
### Tests
|
||||
|
||||
```bash
|
||||
PYTHONPATH=agent_framework/src pytest -q tests/unit
|
||||
```
|
||||
|
||||
Result validated in this generation:
|
||||
|
||||
```text
|
||||
17 passed
|
||||
```
|
||||
|
||||
### Security
|
||||
|
||||
The `.env` files were sanitized so they do not contain real keys. Configure your credentials locally before using OCI/Langfuse.
|
||||
|
||||
### Delta to FIRST standard
|
||||
|
||||
> Content consolidated from `Documentacao/README_FIRST_ENTERPRISE_DELTA.md`.
|
||||
|
||||
This version fixes the priorities identified in the comparison with FIRST:
|
||||
|
||||
1. Real Oracle Session Repository
|
||||
2. Real Oracle Message History
|
||||
3. Real Oracle LangGraph Checkpoint Repository
|
||||
4. LangGraph Deep Telemetry
|
||||
5. Token Accounting
|
||||
6. Cost Accounting
|
||||
7. SSE Session Lock
|
||||
8. SSE Replay Buffer
|
||||
9. SSE KeepAlive
|
||||
10. Recovery through Last-Event-ID
|
||||
11. Redis Provider and Distributed Cache
|
||||
12. Oracle Vector Provider
|
||||
13. Oracle Graph Provider
|
||||
14. RAG Telemetry
|
||||
15. Langfuse Generation Tracking
|
||||
16. Compatible OpenTelemetry/Event Bus
|
||||
17. Preserved OCI Streaming Exporter
|
||||
|
||||
Domain logic remains generic; the framework does not copy FIRST-specific billing rules.
|
||||
|
||||
### Maximum operations and accounting
|
||||
|
||||
> Content consolidated from `Documentacao/README_MAX_OPERACIONAL.md`.
|
||||
|
||||
This version adds the operational adjustments that were missing to bring the framework closer to the FIRST production standard.
|
||||
|
||||
### Adjustments included in this version
|
||||
|
||||
### 1. Langfuse Enterprise Adapter
|
||||
|
||||
New module:
|
||||
|
||||
```text
|
||||
agent_framework/observability/langfuse_enterprise.py
|
||||
```
|
||||
|
||||
Includes an adapter compatible with Langfuse SDKs v2/v3 for:
|
||||
|
||||
- trace updates;
|
||||
- trace scoring/evaluation;
|
||||
- prompt registry when supported by the SDK;
|
||||
- isolation of Langfuse API differences.
|
||||
|
||||
### 2. Persistent Token and Cost Accounting
|
||||
|
||||
New package:
|
||||
|
||||
```text
|
||||
agent_framework/billing/
|
||||
```
|
||||
|
||||
Includes:
|
||||
|
||||
- `UsageRecord`
|
||||
- `SQLiteUsageRepository`
|
||||
- `OracleUsageRepository`
|
||||
- `create_usage_repository(settings)`
|
||||
|
||||
The LLM provider now records automatically:
|
||||
|
||||
- `prompt_tokens`
|
||||
- `completion_tokens`
|
||||
- `cached_tokens`
|
||||
- `total_tokens`
|
||||
- `cost_usd`
|
||||
- `cost_brl`
|
||||
- `tenant_id`
|
||||
- `agent_id`
|
||||
- `session_id`
|
||||
- `message_id`
|
||||
|
||||
New endpoint:
|
||||
|
||||
```http
|
||||
GET /debug/usage
|
||||
GET /debug/usage?tenant_id=default
|
||||
GET /debug/usage?session_id=<id>
|
||||
```
|
||||
|
||||
### 3. Operational RAG Service
|
||||
|
||||
New module:
|
||||
|
||||
```text
|
||||
agent_framework/rag/rag_service.py
|
||||
```
|
||||
|
||||
Includes:
|
||||
|
||||
- `RagService.add_documents()`
|
||||
- `RagService.retrieve()`
|
||||
- `RagResult.as_prompt_context()`
|
||||
- telemetry for latency, document count, top scores, and graph.
|
||||
|
||||
### 4. New configuration
|
||||
|
||||
Variable added:
|
||||
|
||||
```env
|
||||
USAGE_REPOSITORY_PROVIDER=sqlite
|
||||
```
|
||||
|
||||
Values:
|
||||
|
||||
```text
|
||||
sqlite
|
||||
oracle
|
||||
autonomous
|
||||
```
|
||||
|
||||
### 5. Local operational compatibility
|
||||
|
||||
By default, usage accounting uses SQLite even when everything else is in memory. This makes it possible to test locally without Oracle.
|
||||
|
||||
### Quick test
|
||||
|
||||
```bash
|
||||
cd agent_template_backend
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
Test a message:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/gateway/message \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"channel":"web","payload":{"text":"teste","user_id":"u1","session_id":"s1"}}'
|
||||
```
|
||||
|
||||
Check usage/cost:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/debug/usage
|
||||
```
|
||||
|
||||
### To run closer to a production pattern
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=sqlite
|
||||
MEMORY_REPOSITORY_PROVIDER=sqlite
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=sqlite
|
||||
USAGE_REPOSITORY_PROVIDER=sqlite
|
||||
CACHE_BACKEND_PROVIDER=sqlite
|
||||
VECTOR_STORE_PROVIDER=sqlite
|
||||
ENABLE_LANGFUSE=true
|
||||
LANGFUSE_HOST=http://localhost:3000
|
||||
LANGFUSE_PUBLIC_KEY=...
|
||||
LANGFUSE_SECRET_KEY=...
|
||||
```
|
||||
|
||||
For Autonomous Database:
|
||||
|
||||
```env
|
||||
SESSION_REPOSITORY_PROVIDER=oracle
|
||||
MEMORY_REPOSITORY_PROVIDER=oracle
|
||||
CHECKPOINT_REPOSITORY_PROVIDER=oracle
|
||||
USAGE_REPOSITORY_PROVIDER=oracle
|
||||
CACHE_BACKEND_PROVIDER=oracle
|
||||
VECTOR_STORE_PROVIDER=oracle
|
||||
GRAPH_STORE_PROVIDER=oracle
|
||||
ADB_USER=...
|
||||
ADB_PASSWORD=...
|
||||
ADB_DSN=...
|
||||
ADB_WALLET_LOCATION=...
|
||||
ADB_TABLE_PREFIX=AGENTFW
|
||||
```
|
||||
|
||||
### Complementary supervisor validation
|
||||
|
||||
> Content consolidated from `docs/docs_GLOBAL_SUPERVISOR_VALIDATION.txt`.
|
||||
|
||||
VALIDATION - GLOBAL SUPERVISOR
|
||||
|
||||
Implemented changes:
|
||||
|
||||
1. Framework
|
||||
- agent_framework.global_supervisor.models
|
||||
- agent_framework.global_supervisor.config
|
||||
- agent_framework.global_supervisor.session_store
|
||||
- agent_framework.global_supervisor.router
|
||||
- agent_framework.global_supervisor.client
|
||||
|
||||
2. New service
|
||||
- agent_gateway/app/main.py
|
||||
- agent_gateway/app/settings.py
|
||||
- agent_gateway/config/backends.yaml
|
||||
- agent_gateway/README.md
|
||||
- agent_gateway/Dockerfile
|
||||
- agent_gateway/docs/ARQUITETURA_GLOBAL_SUPERVISOR.md
|
||||
|
||||
3. Docker Compose
|
||||
- agent-gateway service added on port 8010.
|
||||
|
||||
Validations performed:
|
||||
|
||||
- python3 -m compileall -q agent_framework/src/agent_framework/global_supervisor agent_gateway/app
|
||||
Result: OK
|
||||
|
||||
- Hybrid-routing smoke test:
|
||||
Input 1: "My bill is too high" -> billing
|
||||
Input 2: "and this amount?" on the same session_id -> billing via active_backend
|
||||
Result: OK
|
||||
|
||||
- FastAPI app import smoke test:
|
||||
from app.main import app, registry, router
|
||||
Result: OK
|
||||
|
||||
Note:
|
||||
- The gateway SSE proxy was left as a future step. The `/gateway/message/sse` endpoint already routes and forwards as a normal message; for end-to-end SSE, a proxy from `/gateway/events/{session_id}` to the active backend can be implemented.
|
||||
|
||||
### Source files
|
||||
|
||||
The files below were consolidated into this manual:
|
||||
|
||||
- `Documentacao/README_FIRST_READY.md`
|
||||
- `Documentacao/README_FIRST_ENTERPRISE_PLUS.md`
|
||||
- `Documentacao/README_FIRST_ENTERPRISE_DELTA.md`
|
||||
- `Documentacao/README_MAX_OPERACIONAL.md`
|
||||
- `docs/docs_GLOBAL_SUPERVISOR_VALIDATION.txt`
|
||||
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
@@ -0,0 +1,157 @@
|
||||
# 12 — Input Guardrail Feedback and Blocked-Turn Semantics
|
||||
|
||||
## Goal
|
||||
|
||||
This document describes how `AgentWorkflow`, implemented in `app/workflows/agent_graph.py`, should handle a turn interrupted by an input guardrail without turning every interruption into a generic “security rule” message.
|
||||
|
||||
The core rule is to keep three concerns separate:
|
||||
|
||||
1. **the guardrail technical decision**, used by the runtime and observability;
|
||||
2. **the user-facing message**, appropriate to the type of block or clarification need;
|
||||
3. **the turn state**, which must not carry routing, tool, or judge data from a turn that was interrupted before those stages.
|
||||
|
||||
## Expected flow
|
||||
|
||||
```text
|
||||
user message
|
||||
↓
|
||||
input_guardrails
|
||||
↓
|
||||
allowed?
|
||||
├─ yes → routing → tools/agent → composition → output_guardrails
|
||||
│
|
||||
└─ no
|
||||
↓
|
||||
select public handling
|
||||
↓
|
||||
clear routing/tools/judges state for this turn
|
||||
↓
|
||||
build a safe user-facing message
|
||||
↓
|
||||
output_guardrails
|
||||
↓
|
||||
persistence/response
|
||||
```
|
||||
|
||||
A blocking input guardrail must be decided **before any side-effecting tool is executed**.
|
||||
|
||||
## Internal `reason` is not the user response
|
||||
|
||||
The `reason` field should remain available to logs, traces, events, and diagnostics. It should not be exposed verbatim when it may reveal internal mechanisms or when the technical wording is not appropriate for the end user.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
COER.reason = "utterance is incomprehensible or contains an ambiguous negation"
|
||||
```
|
||||
|
||||
A public response may be:
|
||||
|
||||
```text
|
||||
"I could not fully understand your last message because it seems incomplete or ambiguous. Could you rephrase or complete what you meant?"
|
||||
```
|
||||
|
||||
## Handling by guardrail type
|
||||
|
||||
Exact behavior remains configurable, but the expected semantics are:
|
||||
|
||||
| Guardrail | Recommended public handling |
|
||||
|---|---|
|
||||
| `COER` | ask for clarification/rephrasing; do not frame ordinary ambiguity as a security incident |
|
||||
| `PINJ` | block safely without describing the internal mechanism |
|
||||
| `DLEX_IN` | block or request reformulation without exposing internal/sensitive data |
|
||||
| `INPUT_SIZE` | ask the user to reduce the input |
|
||||
| `TOX` | apply the configured policy for inappropriate content |
|
||||
| `CMP` | respond according to the compliance policy |
|
||||
| unknown | use a safe generic fallback |
|
||||
|
||||
## Clearing blocked-turn state
|
||||
|
||||
When input is blocked before routing, the final state for that turn must not reuse residual data from the previous turn.
|
||||
|
||||
At a minimum, the workflow should avoid presenting these as current:
|
||||
|
||||
```text
|
||||
route_decision
|
||||
mcp_tools
|
||||
mcp_results
|
||||
judge_results
|
||||
```
|
||||
|
||||
Metadata should clearly indicate that the turn was interrupted at the input-guardrail stage.
|
||||
|
||||
This prevents misleading diagnostics such as:
|
||||
|
||||
```text
|
||||
route = blocked
|
||||
mcp_results = [tool executed]
|
||||
```
|
||||
|
||||
when the tool result actually belongs to the previous turn.
|
||||
|
||||
## The public message also goes through output guardrails
|
||||
|
||||
A response created because of an input block is still agent output. Therefore it should follow the same output-validation pipeline before reaching the user.
|
||||
|
||||
This allows `DLEX_OUT`, `PINJ`, `TOXOUT`, Output Supervisor, and other policies to remove or sanitize information that should not be exposed.
|
||||
|
||||
## Relationship with `agent_graph.py`
|
||||
|
||||
This feature belongs to template orchestration because it defines precedence between graph nodes and blocked-turn state semantics.
|
||||
|
||||
When changing `app/workflows/agent_graph.py`, preserve these invariants:
|
||||
|
||||
- `input_guardrails` runs before routing/tools;
|
||||
- an input block does not execute a transactional action after the block;
|
||||
- the public response is not the raw guardrail `reason`;
|
||||
- residual routing/tools/judges state does not survive as the blocked turn result;
|
||||
- the public message passes through `output_guardrails` before persistence/response.
|
||||
|
||||
The same semantics must be preserved in the official templates and equivalent variants under `Tuning-Performance`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### The user receives “I could not continue because of a security rule” for a merely incomplete phrase
|
||||
|
||||
Check:
|
||||
|
||||
1. which guardrail returned `allowed=false`;
|
||||
2. whether `COER` is handled as clarification rather than a generic security block;
|
||||
3. whether the blocked branch builds a guardrail-specific public message;
|
||||
4. whether the generic fallback is used only when no specific handling exists.
|
||||
|
||||
### Metadata shows a tool as executed while `route=blocked`
|
||||
|
||||
Check whether the blocked branch clears transient turn state before returning. Also confirm that the tool was not executed in the same turn before input-guardrail evaluation.
|
||||
|
||||
### The block response exposes internal details
|
||||
|
||||
Do not use `reason` directly as user-facing text. Generate the public message and keep `reason` for observability only.
|
||||
|
||||
### The block response skips output guardrails
|
||||
|
||||
Check the graph edge. The expected path is:
|
||||
|
||||
```text
|
||||
input_guardrails blocked
|
||||
→ build public response
|
||||
→ output_guardrails
|
||||
→ persist
|
||||
```
|
||||
|
||||
not:
|
||||
|
||||
```text
|
||||
input_guardrails blocked
|
||||
→ persist
|
||||
```
|
||||
|
||||
## Recommended regression tests
|
||||
|
||||
Cover at least:
|
||||
|
||||
- `COER=false` asks for clarification instead of returning a generic security message;
|
||||
- blocked branch does not retain previous-turn `mcp_results`/routing;
|
||||
- no transactional tool executes after an input block;
|
||||
- the public message passes through output guardrails;
|
||||
- an unknown guardrail still has a safe generic fallback.
|
||||
143
agent_framework_oci/docs/developer/en/INDEX_DEVELOPER_GUIDE.md
Normal file
143
agent_framework_oci/docs/developer/en/INDEX_DEVELOPER_GUIDE.md
Normal file
@@ -0,0 +1,143 @@
|
||||
### Developer Index — Agent Framework OCI
|
||||
|
||||
### How to use this documentation
|
||||
|
||||
The documentation has three clear levels:
|
||||
|
||||
1. **Main tutorial:** [`README_en.md`](README_en.md) — creation, configuration, execution, and testing of an agent from start to finish.
|
||||
2. **Architecture:** [01 — Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) — components, responsibilities, and where to implement each concern.
|
||||
3. **Specialized references:** manuals `02` through `12` — in-depth implementation and troubleshooting by capability.
|
||||
|
||||
If you are starting a new agent, begin with `README_en.md`.
|
||||
|
||||
If something is not working, use **Search by problem** below.
|
||||
|
||||
### Search by problem
|
||||
|
||||
| Problem / question | What is usually involved | Where to look |
|
||||
|---|---|---|
|
||||
| The framework does not find the correct agent/intent | routing, intents, threshold, deterministic/LLM mode | [Routing and Stickiness](docs/developer/en/02_routing_stickiness_and_intent_shift.md) |
|
||||
| The agent gets stuck on the same subject and does not change intent | route stickiness, intent shift, handoff | [Routing and Stickiness](docs/developer/en/02_routing_stickiness_and_intent_shift.md) |
|
||||
| An answer that should fill a parameter is interpreted as a new intent | transactional precedence, parameter extraction | [Transactional Workflows](docs/developer/en/03_transaction_workflows_and_state.md) |
|
||||
| The transaction keeps asking for the same parameter | transaction state, extractor, schema | [Transactional Workflows](docs/developer/en/03_transaction_workflows_and_state.md) and [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) |
|
||||
| “yes/no” confirmation does not continue the flow | confirmation state, transaction state | [Transactional Workflows](docs/developer/en/03_transaction_workflows_and_state.md) |
|
||||
| A completed transaction reappears | old checkpoint versus active transaction state | [Transactional Workflows](docs/developer/en/03_transaction_workflows_and_state.md) and [LTM/Checkpoint](docs/developer/en/08_long_term_memory_and_checkpoint.md) |
|
||||
| The system says it executed something, but there is no evidence | MCP result, `COMPLETED` state, transactional judges | [Transactional Workflows](docs/developer/en/03_transaction_workflows_and_state.md) and [Guardrails/Judges](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| A tool does not appear or cannot be found | `tools.yaml`, MCP catalog, discovery | [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) |
|
||||
| MCP Server does not appear in the catalog | registration, manifest/discovery, MCP Gateway | [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) and [Gateways](docs/developer/en/05_agent_gateway_mcp_gateway_and_auth.md) |
|
||||
| Parameters sent to the tool are wrong | schema, mapping, BusinessContext, extractor | [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) |
|
||||
| A transactional operation executes without confirmation | tool policy, `require_confirmation` | [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) |
|
||||
| A name search requires an overly exact match | parameter extraction/mapping and agent logic | [MCP/Tools](docs/developer/en/04_mcp_integration_tools_and_policies.md) |
|
||||
| I receive 401 between gateway/backend/MCP | Basic Auth, credentials per hop | [Gateways and Auth](docs/developer/en/05_agent_gateway_mcp_gateway_and_auth.md) |
|
||||
| I need to decide whether something belongs to the framework or the agent | core/agent boundary | [Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) |
|
||||
| An agent-specific guardrail is breaking another agent | extensibility, domain imports in the core | [Guardrails and Judges](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| An incomplete phrase receives a generic “security rule” message | input-guardrail feedback, `COER`, blocked-turn state | [Input Guardrail Feedback](./12_input_guardrail_feedback_and_blocked_turns.md) |
|
||||
| `route=blocked` appears together with tools/results from another turn | blocked-turn state cleanup | [Input Guardrail Feedback](./12_input_guardrail_feedback_and_blocked_turns.md) |
|
||||
| A judge does not run in a transaction | sampling, `always_run_for_transactional`, transaction signals | [Guardrails and Judges](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| Workflow completes and generates a protocol, but the final response becomes a safety message | `expected_protocols`, `CMP`, `DLEX_OUT`, `output_guardrails` ordering | [Guardrails and Judges](./06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| Groundedness is evaluating without the correct context | RAG context, MCP evidence, judge inputs | [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
| RAG does not find content | provider, ingestion, embeddings, configuration | [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
| I do not know whether to use RAG, memory, or a tool | separation of responsibilities | [Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) and [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
| Memory disappears when changing sessions | LTM versus conversation memory | [LTM and Checkpoint](docs/developer/en/08_long_term_memory_and_checkpoint.md) |
|
||||
| Memory from one customer/agent appears in another | identity key, tenant/agent/customer isolation | [LTM and Checkpoint](docs/developer/en/08_long_term_memory_and_checkpoint.md) |
|
||||
| I need to retrieve `reasoning_content` | `ainvoke_response()` | [LLM Rich Response](docs/developer/en/09_llm_rich_response_reasoning.md) |
|
||||
| `reasoning_content` is `None` | provider/model does not expose the field | [LLM Rich Response](docs/developer/en/09_llm_rich_response_reasoning.md) |
|
||||
| There are unnecessary LLM calls | deterministic routing, concurrency, cache | [Performance](docs/developer/en/10_performance_cache_and_async_runtime.md) |
|
||||
| There is a deadlock or wait across event loops | cross-loop sequence/runtime | [Performance](docs/developer/en/10_performance_cache_and_async_runtime.md) |
|
||||
| Logs/traces do not correlate the same agent | labels, IDs, and observability mapping | [Observability](docs/developer/en/11_observability_persistence_and_operational_readiness.md) |
|
||||
| Sequence is interfering with processing | asynchronous sequence implementation | [Observability](docs/developer/en/11_observability_persistence_and_operational_readiness.md) and [Performance](docs/developer/en/10_performance_cache_and_async_runtime.md) |
|
||||
| An old example does not compile | historical documentation versus current API | [README vs Code Validation](docs/developer/en/VALIDATION_README_ALIGNMENT.md) |
|
||||
| I need to create a new agent from scratch | complete flow | [`README_en.md`](README_en.md) |
|
||||
| I need to know where to place a new feature | architecture and boundaries | [Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) |
|
||||
|
||||
### Search by feature
|
||||
|
||||
### [01 — Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md)
|
||||
|
||||
**What it is:** overview of components, contracts, and responsibility boundaries.
|
||||
|
||||
**Use when:** you need to understand the platform, decide where to implement something, or avoid coupling between core and agent.
|
||||
|
||||
### [02 — Routing, Route Stickiness, and Intent Shift](docs/developer/en/02_routing_stickiness_and_intent_shift.md)
|
||||
|
||||
**What it is:** complete reference for agent/intent discovery, stickiness, handoff, and intent changes.
|
||||
|
||||
**Use when:** the message goes to the wrong agent, does not change intent, or loses continuity.
|
||||
|
||||
### [03 — Transactional Workflows and State](docs/developer/en/03_transaction_workflows_and_state.md)
|
||||
|
||||
**What it is:** multi-turn transaction lifecycle, states, confirmation, pause/resume, and operational evidence.
|
||||
|
||||
**Use when:** there are loops, incorrect confirmations, incorrect resumes, or critical operations.
|
||||
|
||||
### [04 — MCP, Tools, Policies, and Parameter Extraction](docs/developer/en/04_mcp_integration_tools_and_policies.md)
|
||||
|
||||
**What it is:** reference for tools, MCP Servers, mappings, policies, and parameter extraction.
|
||||
|
||||
**Use when:** tool integration/execution is incorrect or needs to be created.
|
||||
|
||||
### [05 — Agent Gateway, MCP Gateway, and Authentication](docs/developer/en/05_agent_gateway_mcp_gateway_and_auth.md)
|
||||
|
||||
**What it is:** gateway responsibilities, governance, and authentication between components.
|
||||
|
||||
**Use when:** there is an ingress, catalog, authorization, 401, or gateway deployment problem.
|
||||
|
||||
### [06 — Guardrails, Judges, and Transaction Evaluation](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md)
|
||||
|
||||
**What it is:** native/external validations, judges, grounding, and rules for transactional turns.
|
||||
|
||||
**Use when:** a validation blocks, does not run, or produces an incorrect evaluation.
|
||||
|
||||
### [07 — RAG, BusinessContext, and Grounding](docs/developer/en/07_rag_business_context_and_grounding.md)
|
||||
|
||||
**What it is:** RAG providers, retrieved context, BusinessContext, and grounding.
|
||||
|
||||
**Use when:** retrieved knowledge does not correctly reach the agent/judge.
|
||||
|
||||
### [08 — Long-Term Memory and Checkpoint](docs/developer/en/08_long_term_memory_and_checkpoint.md)
|
||||
|
||||
**What it is:** durable memory, conversation memory, identity, and state snapshots.
|
||||
|
||||
**Use when:** context disappears, leaks, or the workflow resumes from the wrong place.
|
||||
|
||||
### [09 — LLM Rich Response and reasoning_content](docs/developer/en/09_llm_rich_response_reasoning.md)
|
||||
|
||||
**What it is:** structured inference response beyond the `str` returned by `ainvoke()`.
|
||||
|
||||
**Use when:** consumers need metadata, usage, or reasoning exposed by the provider.
|
||||
|
||||
### [10 — Performance, Cache, and Async Runtime](docs/developer/en/10_performance_cache_and_async_runtime.md)
|
||||
|
||||
**What it is:** concurrency, cache, LLM, and event-loop optimizations.
|
||||
|
||||
**Use when:** there is avoidable latency, serial processing, or deadlock.
|
||||
|
||||
### [11 — Observability, Persistence, and Operational Readiness](docs/developer/en/11_observability_persistence_and_operational_readiness.md)
|
||||
|
||||
**What it is:** correlation, events, labels, sequence, persistence, and diagnostics.
|
||||
|
||||
**Use when:** it is necessary to prove the executed path or diagnose production.
|
||||
|
||||
### [12 — Input Guardrail Feedback and Blocked-Turn Semantics](./12_input_guardrail_feedback_and_blocked_turns.md)
|
||||
|
||||
**What it is:** public handling of input blocks, blocked-turn state cleanup, and output-guardrail validation of generated feedback.
|
||||
|
||||
**Use when:** block messages are generic, `COER` should ask for clarification, or blocked-turn metadata contains stale routing/tool results.
|
||||
|
||||
### Main tutorial
|
||||
|
||||
[`README_en.md`](README_en.md) remains the reference for the complete step-by-step flow:
|
||||
|
||||
`architecture → configuration → agent creation → registration → state → routing → tools → MCP → identity → execution → tests → gateways → memory → RAG`.
|
||||
|
||||
### Maintenance
|
||||
|
||||
Do not create another tutorial in parallel with `README_en.md`.
|
||||
|
||||
When evolving a feature:
|
||||
|
||||
- update the README only if the normal development flow changed;
|
||||
- update the specialized manual with behavior, configuration, examples, and troubleshooting;
|
||||
- update SPECs if the contract changed;
|
||||
- keep release notes as history, not as the only current documentation.
|
||||
|
||||
@@ -0,0 +1,83 @@
|
||||
### Documentation Alignment Validation
|
||||
|
||||
### Goal
|
||||
|
||||
Record how the documentation for this version was reorganized and which sources developers should use.
|
||||
|
||||
### Structural decision
|
||||
|
||||
The root `README_en.md` is the **single end-to-end main tutorial**.
|
||||
|
||||
The former `01_architecture_and_agent_development.md` was removed because it repeated a large part of the README, but not all of it. This created ambiguity: two documents appeared to teach the same thing, but one was partial.
|
||||
|
||||
The new structure replaces that file with `01_architecture_and_concepts.md`, which contains only architecture, concepts, responsibilities, and extension criteria.
|
||||
|
||||
### Validation of `README_old2.md`
|
||||
|
||||
`Documentacao/README_old2.md` remains useful as history, but it is not the primary source for development.
|
||||
|
||||
Later evolutions were found in the current README and code, including:
|
||||
|
||||
- SPECs/SDDs;
|
||||
- more complete `llm_profiles.yaml` configuration;
|
||||
- Channel Gateway and canonical contracts;
|
||||
- `memory` and `summary_memory` in the current agent lifecycle;
|
||||
- `prepare_memory_context()` and `build_messages()`;
|
||||
- `RuntimeContext`;
|
||||
- `normalize_tools_by_intent()`;
|
||||
- `build_tool_arguments()`;
|
||||
- `execute_tools_for_intent()`;
|
||||
- transaction-state helpers;
|
||||
- direct MCP responses;
|
||||
- evolution of gateways, RAG, memory, and policies.
|
||||
|
||||
### Correction applied to the main README
|
||||
|
||||
The following typo was corrected in the generated package:
|
||||
|
||||
```python
|
||||
from app.agents.financeiro_agent import FinanceirotAgent
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```python
|
||||
from app.agents.financeiro_agent import FinanceiroAgent
|
||||
```
|
||||
|
||||
The correct class is confirmed by the code and the rest of the documentation.
|
||||
|
||||
### APIs confirmed in the current implementation
|
||||
|
||||
```python
|
||||
AgentRuntimeMixin.get_runtime_context()
|
||||
AgentRuntimeMixin.normalize_tools_by_intent()
|
||||
AgentRuntimeMixin.build_tool_arguments()
|
||||
AgentRuntimeMixin.execute_tools_for_intent()
|
||||
AgentRuntimeMixin.prepare_memory_context()
|
||||
AgentRuntimeMixin.build_messages()
|
||||
AgentRuntimeMixin.transaction_state_patch()
|
||||
AgentRuntimeMixin.transaction_clarification_message()
|
||||
AgentRuntimeMixin.transaction_confirmation_message()
|
||||
AgentRuntimeMixin.build_direct_mcp_answer()
|
||||
```
|
||||
|
||||
### Trust order
|
||||
|
||||
1. code for the version;
|
||||
2. main README for the same version;
|
||||
3. SPECs/SDDs;
|
||||
4. specialized manuals;
|
||||
5. release notes;
|
||||
6. `README_old*` documents.
|
||||
|
||||
### Future maintenance rule
|
||||
|
||||
A feature evolution should update:
|
||||
|
||||
1. the main README, **only if it changes the normal development path**;
|
||||
2. the feature's specialized manual, with technical details, behavior, configuration, and troubleshooting;
|
||||
3. the SPEC, when there is a contract change;
|
||||
4. the release note, when it is necessary to record the historical change.
|
||||
|
||||
Do not create a new “main manual” for a feature. Do not keep functional fixes permanently only in release notes.
|
||||
Reference in New Issue
Block a user