Enterprise-MCP-Service-Bus/RI/docs/How-To.en-us.md

How-To — Running the RI

Practical, task-oriented guide. To understand what each component does and why, see AS-BUILT.md. Target audience: whoever is going to run this RI.

Prerequisites

• Python 3.12+ and pip/uv.
• Optional: opa binary on the PATH (or OPA_BINARY pointing to it) — without it, the tests that exercise the real PDP are skipped (pytest.skip), not failed.
• Optional: docker on the PATH — without it, tests/e2e/test_real_keycloak.py and tests/e2e/test_security_acceptance.py are skipped for the same reason.
• Optional: credentials for an OpenAI-compatible LLM (LLM_BASE_URL/LLM_API_KEY/LLM_MODEL, see .env.example) — without them, tests/e2e/test_example_agent.py and tests/e2e/test_multi_agent_mode_a.py are skipped.

Installation

make install         # creates .venv and installs the package in editable mode (dev extras included)

Installing OPA (without Docker)

OPA is a static Go binary — it doesn't need Docker or a build step:

curl -sL -o /tmp/opa "https://openpolicyagent.org/downloads/latest/opa_linux_amd64_static"
chmod +x /tmp/opa
cp /tmp/opa .venv/bin/opa
opa version

Reference version used in this RI: OPA v1.20.2 (Rego v1 is the default — config/policies/*.rego uses import rego.v1).

Lint, Typecheck, and Schema Validation

make lint             # ruff check + ruff format --check
make typecheck        # mypy --strict
make validate-schemas # validates all YAML under config/ against its Pydantic model
make export-schemas   # regenerates schemas/*.schema.json from the models

Running the Tests

make test             # == pytest --forked

--forked (pytest-forked) isolates each test function in its own process — necessary because several in-process ASGI server pairs, sharing a single event loop across many sequential tests within the same pytest session, produced intermittent async cleanup failures. With --forked, the suite is 100% deterministic across repeated runs. The cost is only execution time (about +1min on this reference machine). make test/CI use --forked by default.

Per suite, what each one proves:

• pytest tests/unit --forked — isolated behavior of each component (schemas, PDP, entitlement, authn, registry, downstream identity, approval, fabric router, audit, canonical request, rest adapter, bypass detection, gateway cache, shared client lint, profile intelligence, offering filter + property invariants, orchestrator, agent runtime, PII redaction, prompt store).
• pytest tests/e2e --forked — end-to-end flows against a real in-process Gateway (test_walking_skeleton.py, test_governed_gateway.py), a real Keycloak/Docker (test_real_keycloak.py), a real LLM (test_example_agent.py, test_multi_agent_mode_a.py), fail policy (test_fail_closed.py).
• pytest tests/contract --forked — the input/output contract of each tool, connecting directly to each domain server (never through the Gateway) — a tool's contract belongs to the domain server, regardless of who is or isn't entitled to call it.
• pytest tests/adversarial --forked — adversarial prompt/tool injection scenarios (test_prompt_injection.py, test_tool_poisoning.py, test_payload_identity_spoofing.py) and protocol/identity scenarios (test_protocol_identity.py).
• pytest tests/e2e/test_security_acceptance.py -v — the formal security acceptance suite, against the real docker-compose.yml (--profile core --profile identity, Keycloak included). It does not run with --forked/make test — it requires Docker and pays the build+Keycloak cost per run, which is why it's run separately:

  pytest tests/e2e/test_security_acceptance.py -v
  

Bringing Up the Local Stack

Without Docker

source .venv/bin/activate
opa run --server --addr 127.0.0.1:8181 config/policies/ &
export SALES_DOMAIN_SERVICE_TOKEN=dev-sales-domain-service-token
export FINANCE_DOMAIN_SERVICE_TOKEN=dev-finance-domain-service-token
python3 -m example_mcp_servers.sales_domain.server &
python3 -m example_mcp_servers.finance_domain.server &
python3 -m emcp_bus.gateway.server

Without OIDC_JWKS_URL/OIDC_ISSUER set, the Gateway runs in "no IdP configured" mode: it denies everything by default (tools/list empty, tools/call always UNAUTHENTICATED) — see interfaces/mcp-gateway.md for the two modes.

With Docker Compose

make up      # == docker compose --profile core up --build          (without IdP)
make down
make logs

Available profiles (docker-compose.yml):

Profile	Services	What it adds
core	opa, sales-domain, finance-domain, gateway	PEP/PDP + the two example domains — minimal functional setup
identity	+ keycloak	real OIDC authentication
observability	+ otel-collector, jaeger	OTLP tracing (Jaeger UI at http://localhost:16686)

With a real Keycloak:

cp .env.example .env   # uncomment the 3 OIDC_* lines
docker compose --profile core --profile identity up --build

All profiles simultaneously:

docker compose --profile core --profile identity --profile observability up -d --build

Running the Example Agent and the Multi-Agent Modes

Requires LLM_BASE_URL/LLM_API_KEY/LLM_MODEL in .env (any OpenAI-compatible endpoint; validated against meta.llama-3.3-70b-instruct via OCI Generative AI).

pytest tests/e2e/test_example_agent.py -v          # single agent, real tool-calling
pytest tests/e2e/test_multi_agent_mode_a.py -v     # Mode A (external Global Supervisor, ADR-024)
pytest tests/e2e/test_real_keycloak.py -k mode_b -v  # Mode B (Orchestrator as sole brain)

Mode B requires Docker (it brings up its own real Keycloak).

Publishing a New Capability

1. Write the YAML manifest at config/capabilities/<domain>/<capability>.yaml, following the CapabilityManifest schema (see interfaces/registry-capability-manifest.md for the full schema, and any file under config/capabilities/*/ as a real example — canonical name <domain>.<capability>, risk_tier, owner, entitlements, backend.service pointing to a backend already registered under config/backends/).
2. If the capability needs a new backend, register it at config/backends/<backend>.yaml (see interfaces/downstream-identity.md).
3. Associate the capability with an existing or new ClientProfile at config/clients/profiles/<profile>.yaml, adding the canonical name to allowed_tools.
4. Run make validate-schemas — it validates the new YAML against its Pydantic model before anything else.
5. Run make export-schemas if the Pydantic model itself changed (not necessary just to add an example YAML to an already-existing schema).
6. Restart the Gateway (or the Docker stack) — the Registry is seeded from config/capabilities/ at boot (registry/seed.py); there is no hot-reload in this RI.

Registering a New MCP Client

1. Register the client in the Keycloak realm (or OIDC equivalent) with the client_credentials grant enabled and an audience mapper for the Gateway's OIDC_AUDIENCE (emcp-gateway by default) — see deploy/keycloak/realm-export.json for the 3 example clients already configured this way.
2. Create config/clients/<client-id>.yaml (ClientRegistration: client_id, profile, environment) pointing to an existing or new ClientProfile under config/clients/profiles/.
3. Run make validate-schemas.
4. Restart the Gateway — client registrations are also loaded at boot (load_yaml_models(config.clients_dir, ClientRegistration)).

Observability

Jaeger (Tracing)

docker compose --profile core --profile observability up -d --build
# UI: http://localhost:16686

Self-Hosted Langfuse (LLM Observability)

docker compose -f docker-compose.yml -f deploy/langfuse/docker-compose.override.yml \
  --profile llm-observability up -d
# Wait for it:
curl http://localhost:3000/api/public/health
# Create a project/API key in the UI (http://localhost:3000) or pre-seed via
# LANGFUSE_INIT_PROJECT_PUBLIC_KEY/LANGFUSE_INIT_PROJECT_SECRET_KEY
# (see deploy/langfuse/docker-compose.override.yml).

Then, point the standard OTel SDK variables to Langfuse's native OTLP endpoint (no code changes needed):

export OTEL_EXPORTER_OTLP_ENDPOINT=http://langfuse-web:3000/api/public/otel
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64(public_key:secret_key)>"

This profile is heavy (6 containers: web/worker/postgres/clickhouse/redis/minio) and opt-in, decoupled from the core/identity/observability profiles.