agent_platform_oci/docs/EXTERNAL_GUARDRAILS_JUDGES_en.md

External guardrails and judges: implementation guide

External extensions keep domain policy in the agent package instead of coupling it to agent_framework. The complete executable reference, including copy-and-paste implementations, is the Portuguese guide; its tagged examples are exercised by tests/test_external_guardrails_judges_documentation.py.

Required contracts

• A guardrail normally extends Guardrail and implements evaluate(text, context) -> RailDecision. Its YAML section selects input, retrieval, tool, or output; policy.on_deny selects block, retry, handover, sanitize, observe, or allow.
• A judge implements evaluate(question, answer, context) -> JudgeResult. Its constructor can receive injected llm and settings, plus threshold, profile_name, fail_closed, max_context_chars, and fallback_on_block.
• Class paths accept package.module:Class (recommended) or package.module.Class; that package must be importable by the backend.
• Synchronous methods run in worker threads; asynchronous methods run on the event loop. Judges execute concurrently while result order follows YAML order.

# guardrails.yaml
fail_fast: true
tool:
  - code: FIN_AMOUNT
    type: external
    class: financial_agent.extensions.guardrails:FinancialAmountRail
    kwargs: {max_amount: 1500.0}
    policy: {on_deny: handover}

# judges.yaml
enabled: true
sample_rate: 1.0
always_run_for_transactional: true
fail_closed: true
judges:
  - name: financial_evidence
    type: external
    class: financial_agent.extensions.judges:FinancialEvidenceJudge
    threshold: 0.8

Current template limitation

agent_template_backend currently builds one guardrail pipeline and one judge pipeline at startup. Per-agent guardrails_config_path and judges_config_path entries in agents.yaml do not by themselves switch pipelines for each request. Until per-profile pipeline selection is implemented by the application, startup YAML is effectively global.

The template also constructs guardrails without llm and judges without llm/settings. Deterministic extensions work, but LLM-dependent extensions require GuardrailPipeline(llm=llm, ...) and JudgePipeline(llm=llm, settings=settings, ...). Guardrail constructors do not receive LLM injection; a configured pipeline exposes it through context["guardrail_llm"] or context["llm"].

Use fail-closed for authorization, safety, and transaction controls. Catch expected external judge failures inside the implementation because an unhandled exception may propagate from concurrent evaluation. Never log prompts, credentials, PII, or financial payloads.

pytest -q tests/test_external_guardrails_judges_documentation.py