mirror of
https://github.com/hoshikawa2/agent_platform_oci.git
synced 2026-09-07 10:13:46 +00:00
Ajustes conforme relatorio de testes 2026-08-27
This commit is contained in:
@@ -557,3 +557,82 @@ The files below were consolidated into this manual:
|
||||
### Maintenance rule
|
||||
|
||||
New fixes or evolutions for this subject should update this consolidated document. Release notes may continue to exist as history, but they should not be required to understand or implement the feature.
|
||||
|
||||
|
||||
## Canonical resolution and domain revalidation before execution
|
||||
|
||||
When pre-validation resolves a user reference to a canonical entity, the framework **must not blindly overwrite the parameter and execute the originally selected tool**. The contract keeps requested, resolved and execution values distinct.
|
||||
|
||||
A domain validator may return `transaction_decision` with `resolved_arguments`, `target_tool`, `action_changed`, `requires_reconfirmation`, and an optional customer-facing `confirmation_message`.
|
||||
|
||||
Responsibilities:
|
||||
|
||||
- **Framework:** preserve the requested arguments, apply only canonical arguments declared by the validator, update the transaction to the effective `target_tool`, honor reconfirmation, and retain the decision in pre-validation evidence.
|
||||
- **Agent/domain:** decide business class, policy and effective tool. The framework must not know rules such as “Youtube Premium is strategic”.
|
||||
- **MCP/backend:** execute the final operation chosen by the domain.
|
||||
|
||||
If canonicalization does not change the action, the current tool may remain valid. If entity resolution changes business class/policy/tool, domain revalidation must happen **before confirmation and execution**. Ambiguous or low-confidence resolution must request clarification instead of silently promoting a candidate.
|
||||
|
||||
### Troubleshooting: resolved_subject is correct but execution receives the original text
|
||||
|
||||
If pre-validation records `resolved_subject="Youtube Premium"` while execution still receives `subject="youtube"`, verify that the validator returns `transaction_decision.resolved_arguments` and that the runtime applies the decision before freezing `pending_tool_call` / `confirmation_snapshot`. If the canonical entity is correct but the final tool is wrong, inspect `transaction_decision.target_tool`; that business reclassification belongs to the domain validator, not to the framework.
|
||||
|
||||
For domains that expose an authoritative business classification in backend detail, revalidation should use that evidence before aggregated categories. In Contas, for example, `invoice_detail.parsed_content` preserves `classe=avulso|estrategico|bundle`, while `billing_analysis` may group the same item into broader sections such as `streaming` or partner services. Canonical entity discovery may use any authorized evidence, but the **business decision** should prioritize the source that preserves the domain classification. If classification evidence conflicts, do not silently change the action; preserve the current operation or request clarification according to the agent policy.
|
||||
|
||||
|
||||
## Semantic transactional confirmation: SIM / NAO / CONTINUAR
|
||||
|
||||
Transactions in `AWAITING_CONFIRMATION` use two layers, in this order:
|
||||
|
||||
1. **Deterministic parser** for explicit confirmations/rejections (`sim`, `não`, `confirmo`, `pode fazer`, etc.). This remains the cheapest and safest path and **does not call an LLM**.
|
||||
2. **LLM semantic fallback** only when the deterministic parser is inconclusive. The fallback reuses the same declarative semantic-classifier engine used by paused workflow `expected_input`, injecting the pending prompt, recent context related to the same topic, and the current user utterance.
|
||||
|
||||
Configuration lives in `config/routing.yaml` under `router.transaction_confirmation.semantic_fallback`:
|
||||
|
||||
```yaml
|
||||
router:
|
||||
transaction_confirmation:
|
||||
semantic_fallback:
|
||||
enabled: true
|
||||
allowed_values: [SIM, NAO, CONTINUAR]
|
||||
confirm_values: [SIM]
|
||||
reject_values: [NAO]
|
||||
continue_values: [CONTINUAR]
|
||||
include_relevant_context: true
|
||||
profile_name: router
|
||||
prompt: |
|
||||
Allowed classes: {{ allowed_values }}
|
||||
Pending prompt:
|
||||
{{ pending_prompt }}
|
||||
Relevant context:
|
||||
{{ relevant_conversation_context }}
|
||||
Current user input:
|
||||
{{ user_input }}
|
||||
```
|
||||
|
||||
`SIM` means an unambiguous acceptance, `NAO` an unambiguous rejection, and `CONTINUAR` means the utterance does not safely confirm or reject the pending action. Example: after `Você confirma o cancelamento do serviço Tamboro Mensal?`, the reply `isso mesmo, pode confirmar` can be classified as `SIM` without hardcoding that exact sentence.
|
||||
|
||||
When semantic confirmation succeeds, the router records:
|
||||
|
||||
```json
|
||||
{
|
||||
"transaction_turn_consumed": true,
|
||||
"transaction_confirmation_decision": "confirm",
|
||||
"transaction_confirmation_source": "semantic"
|
||||
}
|
||||
```
|
||||
|
||||
`AgentRuntime` reuses this routed decision instead of re-running the deterministic parser. The change is additive: existing explicit yes/no inputs continue through the deterministic path with no extra LLM call. Semantic generations are named `transaction.confirmation.semantic_classifier` for observability.
|
||||
|
||||
### Durable interrupt compatibility in pause/resume
|
||||
|
||||
The runtime does not use `snapshot.next` alone to decide whether a workflow is paused. A truthy `next` may represent LangGraph helper work, including framework-generated synthetic nodes such as `__pause` and `__continue`.
|
||||
|
||||
A pause is recognized only from a real interrupt. Depending on the LangGraph/checkpointer version, that interrupt may be exposed through `task.interrupts` or persisted in `snapshot.values["__interrupt__"]`. The runtime supports both shapes and deduplicates the payload when both are present.
|
||||
|
||||
This prevents two false diagnoses:
|
||||
|
||||
- treating `snapshot.next` as `PAUSED` when no real interrupt exists;
|
||||
- treating `next=("<node>__pause",)` as invalid pending work when the real interrupt is persisted under `__interrupt__`.
|
||||
|
||||
For workflows using `expected_input.semantic_classifier`, internal tokens such as `SIM`, `NAO`, and `CONTINUAR` remain resume control values and must not be confused with customer-facing output.
|
||||
|
||||
@@ -57,6 +57,69 @@ This version adds a pragmatic guardrail layer to `agent_framework`, inspired by
|
||||
- `RET_REL` — validates retrieval-chunk relevance using a minimum score.
|
||||
- `TOOL_VAL` — validates MCP/tool name, required arguments, negative values, and allowlist.
|
||||
|
||||
### Contract for authorized protocols in output guardrails
|
||||
|
||||
When a workflow or tool produces a **protocol/reference number that must be shown to the same customer**, the agent integration code must register that value in the output context before output guardrails run:
|
||||
|
||||
```python
|
||||
ctx["expected_protocols"] = [protocol_number]
|
||||
```
|
||||
|
||||
This field is a **framework contract**. It declares that those exact values were produced or validated by the current flow and may therefore be used by output guardrails as authorization evidence.
|
||||
|
||||
Expected flow:
|
||||
|
||||
```text
|
||||
workflow/tool produces protocol
|
||||
↓
|
||||
agent registers it in expected_protocols
|
||||
↓
|
||||
CMP validates that the displayed protocol belongs to the expected values
|
||||
↓
|
||||
DLEX_OUT does not block that protocol merely because it is an identifier
|
||||
↓
|
||||
response may disclose the protocol to the customer
|
||||
```
|
||||
|
||||
Important rules:
|
||||
|
||||
- `expected_protocols` must contain **only protocols actually produced/expected in the current turn or transaction**.
|
||||
- Do not use `expected_protocols` to allow tokens, credentials, arbitrary internal IDs, or third-party data.
|
||||
- Authorization applies only to listed values; any other identifier remains subject to normal `DLEX_OUT` rules.
|
||||
- The value must be propagated **before `output_guardrails`**. Adding it later has no effect.
|
||||
- For transactional responses, keep protocol evidence in the tool/workflow result so `CMP`, `GND`, and observability can correlate the value.
|
||||
|
||||
Example:
|
||||
|
||||
```python
|
||||
result = await execute_workflow(...)
|
||||
protocol_number = result.get("protocol_number") or result.get("protocolo_id")
|
||||
if protocol_number:
|
||||
ctx["expected_protocols"] = [str(protocol_number)]
|
||||
```
|
||||
|
||||
#### Troubleshooting: workflow completed but the response was replaced by a safety message
|
||||
|
||||
Typical symptom:
|
||||
|
||||
```text
|
||||
workflow = COMPLETED
|
||||
CMP = allowed
|
||||
DLEX_OUT = blocked because of "internal protocol"
|
||||
final response = "I could not safely validate this response..."
|
||||
```
|
||||
|
||||
Check, in this order:
|
||||
|
||||
1. Is the generated protocol present in the tool/workflow result or evidence?
|
||||
2. Did the agent propagate the same value in `ctx["expected_protocols"]`?
|
||||
3. Was `expected_protocols` populated before `output_guardrails`?
|
||||
4. Is the protocol shown in the response exactly one of the expected values?
|
||||
5. Is `DLEX_OUT` actually blocking another real issue such as a secret, token, or third-party data?
|
||||
|
||||
If `expected_protocols` is absent, the framework must not assume that an arbitrary textual identifier is safe to disclose.
|
||||
|
||||
|
||||
### Files changed
|
||||
|
||||
- `agent_framework/src/agent_framework/guardrails/rails.py`
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
# 12 — Input Guardrail Feedback and Blocked-Turn Semantics
|
||||
|
||||
## Goal
|
||||
|
||||
This document describes how `AgentWorkflow`, implemented in `app/workflows/agent_graph.py`, should handle a turn interrupted by an input guardrail without turning every interruption into a generic “security rule” message.
|
||||
|
||||
The core rule is to keep three concerns separate:
|
||||
|
||||
1. **the guardrail technical decision**, used by the runtime and observability;
|
||||
2. **the user-facing message**, appropriate to the type of block or clarification need;
|
||||
3. **the turn state**, which must not carry routing, tool, or judge data from a turn that was interrupted before those stages.
|
||||
|
||||
## Expected flow
|
||||
|
||||
```text
|
||||
user message
|
||||
↓
|
||||
input_guardrails
|
||||
↓
|
||||
allowed?
|
||||
├─ yes → routing → tools/agent → composition → output_guardrails
|
||||
│
|
||||
└─ no
|
||||
↓
|
||||
select public handling
|
||||
↓
|
||||
clear routing/tools/judges state for this turn
|
||||
↓
|
||||
build a safe user-facing message
|
||||
↓
|
||||
output_guardrails
|
||||
↓
|
||||
persistence/response
|
||||
```
|
||||
|
||||
A blocking input guardrail must be decided **before any side-effecting tool is executed**.
|
||||
|
||||
## Internal `reason` is not the user response
|
||||
|
||||
The `reason` field should remain available to logs, traces, events, and diagnostics. It should not be exposed verbatim when it may reveal internal mechanisms or when the technical wording is not appropriate for the end user.
|
||||
|
||||
Example:
|
||||
|
||||
```text
|
||||
COER.reason = "utterance is incomprehensible or contains an ambiguous negation"
|
||||
```
|
||||
|
||||
A public response may be:
|
||||
|
||||
```text
|
||||
"I could not fully understand your last message because it seems incomplete or ambiguous. Could you rephrase or complete what you meant?"
|
||||
```
|
||||
|
||||
## Handling by guardrail type
|
||||
|
||||
Exact behavior remains configurable, but the expected semantics are:
|
||||
|
||||
| Guardrail | Recommended public handling |
|
||||
|---|---|
|
||||
| `COER` | ask for clarification/rephrasing; do not frame ordinary ambiguity as a security incident |
|
||||
| `PINJ` | block safely without describing the internal mechanism |
|
||||
| `DLEX_IN` | block or request reformulation without exposing internal/sensitive data |
|
||||
| `INPUT_SIZE` | ask the user to reduce the input |
|
||||
| `TOX` | apply the configured policy for inappropriate content |
|
||||
| `CMP` | respond according to the compliance policy |
|
||||
| unknown | use a safe generic fallback |
|
||||
|
||||
## Clearing blocked-turn state
|
||||
|
||||
When input is blocked before routing, the final state for that turn must not reuse residual data from the previous turn.
|
||||
|
||||
At a minimum, the workflow should avoid presenting these as current:
|
||||
|
||||
```text
|
||||
route_decision
|
||||
mcp_tools
|
||||
mcp_results
|
||||
judge_results
|
||||
```
|
||||
|
||||
Metadata should clearly indicate that the turn was interrupted at the input-guardrail stage.
|
||||
|
||||
This prevents misleading diagnostics such as:
|
||||
|
||||
```text
|
||||
route = blocked
|
||||
mcp_results = [tool executed]
|
||||
```
|
||||
|
||||
when the tool result actually belongs to the previous turn.
|
||||
|
||||
## The public message also goes through output guardrails
|
||||
|
||||
A response created because of an input block is still agent output. Therefore it should follow the same output-validation pipeline before reaching the user.
|
||||
|
||||
This allows `DLEX_OUT`, `PINJ`, `TOXOUT`, Output Supervisor, and other policies to remove or sanitize information that should not be exposed.
|
||||
|
||||
## Relationship with `agent_graph.py`
|
||||
|
||||
This feature belongs to template orchestration because it defines precedence between graph nodes and blocked-turn state semantics.
|
||||
|
||||
When changing `app/workflows/agent_graph.py`, preserve these invariants:
|
||||
|
||||
- `input_guardrails` runs before routing/tools;
|
||||
- an input block does not execute a transactional action after the block;
|
||||
- the public response is not the raw guardrail `reason`;
|
||||
- residual routing/tools/judges state does not survive as the blocked turn result;
|
||||
- the public message passes through `output_guardrails` before persistence/response.
|
||||
|
||||
The same semantics must be preserved in the official templates and equivalent variants under `Tuning-Performance`.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### The user receives “I could not continue because of a security rule” for a merely incomplete phrase
|
||||
|
||||
Check:
|
||||
|
||||
1. which guardrail returned `allowed=false`;
|
||||
2. whether `COER` is handled as clarification rather than a generic security block;
|
||||
3. whether the blocked branch builds a guardrail-specific public message;
|
||||
4. whether the generic fallback is used only when no specific handling exists.
|
||||
|
||||
### Metadata shows a tool as executed while `route=blocked`
|
||||
|
||||
Check whether the blocked branch clears transient turn state before returning. Also confirm that the tool was not executed in the same turn before input-guardrail evaluation.
|
||||
|
||||
### The block response exposes internal details
|
||||
|
||||
Do not use `reason` directly as user-facing text. Generate the public message and keep `reason` for observability only.
|
||||
|
||||
### The block response skips output guardrails
|
||||
|
||||
Check the graph edge. The expected path is:
|
||||
|
||||
```text
|
||||
input_guardrails blocked
|
||||
→ build public response
|
||||
→ output_guardrails
|
||||
→ persist
|
||||
```
|
||||
|
||||
not:
|
||||
|
||||
```text
|
||||
input_guardrails blocked
|
||||
→ persist
|
||||
```
|
||||
|
||||
## Recommended regression tests
|
||||
|
||||
Cover at least:
|
||||
|
||||
- `COER=false` asks for clarification instead of returning a generic security message;
|
||||
- blocked branch does not retain previous-turn `mcp_results`/routing;
|
||||
- no transactional tool executes after an input block;
|
||||
- the public message passes through output guardrails;
|
||||
- an unknown guardrail still has a safe generic fallback.
|
||||
@@ -6,7 +6,7 @@ The documentation has three clear levels:
|
||||
|
||||
1. **Main tutorial:** [`README_en.md`](README_en.md) — creation, configuration, execution, and testing of an agent from start to finish.
|
||||
2. **Architecture:** [01 — Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) — components, responsibilities, and where to implement each concern.
|
||||
3. **Specialized references:** manuals `02` through `11` — in-depth implementation and troubleshooting by capability.
|
||||
3. **Specialized references:** manuals `02` through `12` — in-depth implementation and troubleshooting by capability.
|
||||
|
||||
If you are starting a new agent, begin with `README_en.md`.
|
||||
|
||||
@@ -31,7 +31,10 @@ If something is not working, use **Search by problem** below.
|
||||
| I receive 401 between gateway/backend/MCP | Basic Auth, credentials per hop | [Gateways and Auth](docs/developer/en/05_agent_gateway_mcp_gateway_and_auth.md) |
|
||||
| I need to decide whether something belongs to the framework or the agent | core/agent boundary | [Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) |
|
||||
| An agent-specific guardrail is breaking another agent | extensibility, domain imports in the core | [Guardrails and Judges](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| An incomplete phrase receives a generic “security rule” message | input-guardrail feedback, `COER`, blocked-turn state | [Input Guardrail Feedback](./12_input_guardrail_feedback_and_blocked_turns.md) |
|
||||
| `route=blocked` appears together with tools/results from another turn | blocked-turn state cleanup | [Input Guardrail Feedback](./12_input_guardrail_feedback_and_blocked_turns.md) |
|
||||
| A judge does not run in a transaction | sampling, `always_run_for_transactional`, transaction signals | [Guardrails and Judges](docs/developer/en/06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| Workflow completes and generates a protocol, but the final response becomes a safety message | `expected_protocols`, `CMP`, `DLEX_OUT`, `output_guardrails` ordering | [Guardrails and Judges](./06_guardrails_judges_and_transaction_evaluation.md) |
|
||||
| Groundedness is evaluating without the correct context | RAG context, MCP evidence, judge inputs | [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
| RAG does not find content | provider, ingestion, embeddings, configuration | [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
| I do not know whether to use RAG, memory, or a tool | separation of responsibilities | [Architecture and Concepts](docs/developer/en/01_architecture_and_concepts.md) and [RAG/Grounding](docs/developer/en/07_rag_business_context_and_grounding.md) |
|
||||
@@ -115,6 +118,12 @@ If something is not working, use **Search by problem** below.
|
||||
|
||||
**Use when:** it is necessary to prove the executed path or diagnose production.
|
||||
|
||||
### [12 — Input Guardrail Feedback and Blocked-Turn Semantics](./12_input_guardrail_feedback_and_blocked_turns.md)
|
||||
|
||||
**What it is:** public handling of input blocks, blocked-turn state cleanup, and output-guardrail validation of generated feedback.
|
||||
|
||||
**Use when:** block messages are generic, `COER` should ask for clarification, or blocked-turn metadata contains stale routing/tool results.
|
||||
|
||||
### Main tutorial
|
||||
|
||||
[`README_en.md`](README_en.md) remains the reference for the complete step-by-step flow:
|
||||
@@ -131,3 +140,4 @@ When evolving a feature:
|
||||
- update the specialized manual with behavior, configuration, examples, and troubleshooting;
|
||||
- update SPECs if the contract changed;
|
||||
- keep release notes as history, not as the only current documentation.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user