mirror of
https://github.com/hoshikawa2/agent_platform_oci.git
synced 2026-11-06 03:14:12 +00:00
adjustments: manual v2
This commit is contained in:
46
docs/EXTERNAL_GUARDRAILS_JUDGES_en.md
Normal file
46
docs/EXTERNAL_GUARDRAILS_JUDGES_en.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# External guardrails and judges: implementation guide
|
||||
|
||||
External extensions keep domain policy in the agent package instead of coupling it to `agent_framework`. The complete executable reference, including copy-and-paste implementations, is [the Portuguese guide](EXTERNAL_GUARDRAILS_JUDGES.md); its tagged examples are exercised by `tests/test_external_guardrails_judges_documentation.py`.
|
||||
|
||||
## Required contracts
|
||||
|
||||
- A guardrail normally extends `Guardrail` and implements `evaluate(text, context) -> RailDecision`. Its YAML section selects `input`, `retrieval`, `tool`, or `output`; `policy.on_deny` selects `block`, `retry`, `handover`, `sanitize`, `observe`, or `allow`.
|
||||
- A judge implements `evaluate(question, answer, context) -> JudgeResult`. Its constructor can receive injected `llm` and `settings`, plus `threshold`, `profile_name`, `fail_closed`, `max_context_chars`, and `fallback_on_block`.
|
||||
- Class paths accept `package.module:Class` (recommended) or `package.module.Class`; that package must be importable by the backend.
|
||||
- Synchronous methods run in worker threads; asynchronous methods run on the event loop. Judges execute concurrently while result order follows YAML order.
|
||||
|
||||
```yaml
|
||||
# guardrails.yaml
|
||||
fail_fast: true
|
||||
tool:
|
||||
- code: FIN_AMOUNT
|
||||
type: external
|
||||
class: financial_agent.extensions.guardrails:FinancialAmountRail
|
||||
kwargs: {max_amount: 1500.0}
|
||||
policy: {on_deny: handover}
|
||||
```
|
||||
|
||||
```yaml
|
||||
# judges.yaml
|
||||
enabled: true
|
||||
sample_rate: 1.0
|
||||
always_run_for_transactional: true
|
||||
fail_closed: true
|
||||
judges:
|
||||
- name: financial_evidence
|
||||
type: external
|
||||
class: financial_agent.extensions.judges:FinancialEvidenceJudge
|
||||
threshold: 0.8
|
||||
```
|
||||
|
||||
## Current template limitation
|
||||
|
||||
`agent_template_backend` currently builds one guardrail pipeline and one judge pipeline at startup. Per-agent `guardrails_config_path` and `judges_config_path` entries in `agents.yaml` do not by themselves switch pipelines for each request. Until per-profile pipeline selection is implemented by the application, startup YAML is effectively global.
|
||||
|
||||
The template also constructs guardrails without `llm` and judges without `llm/settings`. Deterministic extensions work, but LLM-dependent extensions require `GuardrailPipeline(llm=llm, ...)` and `JudgePipeline(llm=llm, settings=settings, ...)`. Guardrail constructors do not receive LLM injection; a configured pipeline exposes it through `context["guardrail_llm"]` or `context["llm"]`.
|
||||
|
||||
Use fail-closed for authorization, safety, and transaction controls. Catch expected external judge failures inside the implementation because an unhandled exception may propagate from concurrent evaluation. Never log prompts, credentials, PII, or financial payloads.
|
||||
|
||||
```bash
|
||||
pytest -q tests/test_external_guardrails_judges_documentation.py
|
||||
```
|
||||
Reference in New Issue
Block a user