mirror of
https://github.com/mcloh/Enterprise-MCP-Service-Bus.git
synced 2026-11-05 22:14:11 +00:00
revisão e complementação da documentação do projeto
This commit is contained in:
Marcos Lohmann
205
RI/docs/How-To.en-us.md
Normal file
205
RI/docs/How-To.en-us.md
Normal file
@@ -0,0 +1,205 @@
|
||||
# How-To — Running the RI
|
||||
|
||||
Practical, task-oriented guide. To understand *what* each component does and *why*, see
|
||||
[`AS-BUILT.md`](AS-BUILT.md). Target audience: whoever is going to **run** this RI.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Python 3.12+ and `pip`/`uv`.
|
||||
- Optional: `opa` binary on the PATH (or `OPA_BINARY` pointing to it) — without it, the tests that
|
||||
exercise the real PDP are skipped (`pytest.skip`), not failed.
|
||||
- Optional: `docker` on the PATH — without it, `tests/e2e/test_real_keycloak.py` and
|
||||
`tests/e2e/test_security_acceptance.py` are skipped for the same reason.
|
||||
- Optional: credentials for an OpenAI-compatible LLM (`LLM_BASE_URL`/`LLM_API_KEY`/`LLM_MODEL`,
|
||||
see `.env.example`) — without them, `tests/e2e/test_example_agent.py` and
|
||||
`tests/e2e/test_multi_agent_mode_a.py` are skipped.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
make install # creates .venv and installs the package in editable mode (dev extras included)
|
||||
```
|
||||
|
||||
### Installing OPA (without Docker)
|
||||
|
||||
OPA is a static Go binary — it doesn't need Docker or a build step:
|
||||
|
||||
```bash
|
||||
curl -sL -o /tmp/opa "https://openpolicyagent.org/downloads/latest/opa_linux_amd64_static"
|
||||
chmod +x /tmp/opa
|
||||
cp /tmp/opa .venv/bin/opa
|
||||
opa version
|
||||
```
|
||||
|
||||
Reference version used in this RI: OPA v1.20.2 (Rego v1 is the default — `config/policies/*.rego`
|
||||
uses `import rego.v1`).
|
||||
|
||||
## Lint, Typecheck, and Schema Validation
|
||||
|
||||
```bash
|
||||
make lint # ruff check + ruff format --check
|
||||
make typecheck # mypy --strict
|
||||
make validate-schemas # validates all YAML under config/ against its Pydantic model
|
||||
make export-schemas # regenerates schemas/*.schema.json from the models
|
||||
```
|
||||
|
||||
## Running the Tests
|
||||
|
||||
```bash
|
||||
make test # == pytest --forked
|
||||
```
|
||||
|
||||
`--forked` (pytest-forked) isolates each test function in its own process — necessary because
|
||||
several in-process ASGI server pairs, sharing a single event loop across many sequential tests
|
||||
within the same pytest session, produced intermittent async cleanup failures. With `--forked`,
|
||||
the suite is 100% deterministic across repeated runs. The cost is only execution time (about
|
||||
+1min on this reference machine). `make test`/CI use `--forked` by default.
|
||||
|
||||
Per suite, what each one proves:
|
||||
|
||||
- `pytest tests/unit --forked` — isolated behavior of each component (schemas, PDP,
|
||||
entitlement, authn, registry, downstream identity, approval, fabric router, audit, canonical
|
||||
request, rest adapter, bypass detection, gateway cache, shared client lint, profile
|
||||
intelligence, offering filter + property invariants, orchestrator, agent runtime, PII
|
||||
redaction, prompt store).
|
||||
- `pytest tests/e2e --forked` — end-to-end flows against a real in-process Gateway
|
||||
(`test_walking_skeleton.py`, `test_governed_gateway.py`), a real Keycloak/Docker
|
||||
(`test_real_keycloak.py`), a real LLM (`test_example_agent.py`, `test_multi_agent_mode_a.py`),
|
||||
fail policy (`test_fail_closed.py`).
|
||||
- `pytest tests/contract --forked` — the input/output contract of each tool, connecting directly
|
||||
to each domain server (never through the Gateway) — a tool's contract belongs to the domain
|
||||
server, regardless of who is or isn't entitled to call it.
|
||||
- `pytest tests/adversarial --forked` — adversarial prompt/tool injection scenarios
|
||||
(`test_prompt_injection.py`, `test_tool_poisoning.py`, `test_payload_identity_spoofing.py`) and
|
||||
protocol/identity scenarios (`test_protocol_identity.py`).
|
||||
- `pytest tests/e2e/test_security_acceptance.py -v` — the formal security acceptance suite,
|
||||
against the real `docker-compose.yml` (`--profile core --profile identity`, Keycloak included).
|
||||
It **does not** run with `--forked`/`make test` — it requires Docker and pays the
|
||||
build+Keycloak cost per run, which is why it's run separately:
|
||||
```bash
|
||||
pytest tests/e2e/test_security_acceptance.py -v
|
||||
```
|
||||
|
||||
## Bringing Up the Local Stack
|
||||
|
||||
### Without Docker
|
||||
|
||||
```bash
|
||||
source .venv/bin/activate
|
||||
opa run --server --addr 127.0.0.1:8181 config/policies/ &
|
||||
export SALES_DOMAIN_SERVICE_TOKEN=dev-sales-domain-service-token
|
||||
export FINANCE_DOMAIN_SERVICE_TOKEN=dev-finance-domain-service-token
|
||||
python3 -m example_mcp_servers.sales_domain.server &
|
||||
python3 -m example_mcp_servers.finance_domain.server &
|
||||
python3 -m emcp_bus.gateway.server
|
||||
```
|
||||
|
||||
Without `OIDC_JWKS_URL`/`OIDC_ISSUER` set, the Gateway runs in "no IdP configured" mode: it
|
||||
denies everything by default (`tools/list` empty, `tools/call` always `UNAUTHENTICATED`) — see
|
||||
[`interfaces/mcp-gateway.md`](interfaces/mcp-gateway.md#dois-modos-de-deployment) for the two
|
||||
modes.
|
||||
|
||||
### With Docker Compose
|
||||
|
||||
```bash
|
||||
make up # == docker compose --profile core up --build (without IdP)
|
||||
make down
|
||||
make logs
|
||||
```
|
||||
|
||||
Available profiles (`docker-compose.yml`):
|
||||
|
||||
| Profile | Services | What it adds |
|
||||
|---|---|---|
|
||||
| `core` | `opa`, `sales-domain`, `finance-domain`, `gateway` | PEP/PDP + the two example domains — minimal functional setup |
|
||||
| `identity` | + `keycloak` | real OIDC authentication |
|
||||
| `observability` | + `otel-collector`, `jaeger` | OTLP tracing (Jaeger UI at `http://localhost:16686`) |
|
||||
|
||||
With a real Keycloak:
|
||||
|
||||
```bash
|
||||
cp .env.example .env # uncomment the 3 OIDC_* lines
|
||||
docker compose --profile core --profile identity up --build
|
||||
```
|
||||
|
||||
All profiles simultaneously:
|
||||
|
||||
```bash
|
||||
docker compose --profile core --profile identity --profile observability up -d --build
|
||||
```
|
||||
|
||||
## Running the Example Agent and the Multi-Agent Modes
|
||||
|
||||
Requires `LLM_BASE_URL`/`LLM_API_KEY`/`LLM_MODEL` in `.env` (any OpenAI-compatible endpoint;
|
||||
validated against `meta.llama-3.3-70b-instruct` via OCI Generative AI).
|
||||
|
||||
```bash
|
||||
pytest tests/e2e/test_example_agent.py -v # single agent, real tool-calling
|
||||
pytest tests/e2e/test_multi_agent_mode_a.py -v # Mode A (external Global Supervisor, ADR-024)
|
||||
pytest tests/e2e/test_real_keycloak.py -k mode_b -v # Mode B (Orchestrator as sole brain)
|
||||
```
|
||||
|
||||
Mode B requires Docker (it brings up its own real Keycloak).
|
||||
|
||||
## Publishing a New Capability
|
||||
|
||||
1. Write the YAML manifest at `config/capabilities/<domain>/<capability>.yaml`, following the
|
||||
`CapabilityManifest` schema (see [`interfaces/registry-capability-manifest.md`](interfaces/registry-capability-manifest.md)
|
||||
for the full schema, and any file under `config/capabilities/*/` as a real example —
|
||||
canonical name `<domain>.<capability>`, `risk_tier`, `owner`, `entitlements`, `backend.service`
|
||||
pointing to a backend already registered under `config/backends/`).
|
||||
2. If the capability needs a new backend, register it at `config/backends/<backend>.yaml`
|
||||
(see [`interfaces/downstream-identity.md`](interfaces/downstream-identity.md)).
|
||||
3. Associate the capability with an existing or new `ClientProfile` at
|
||||
`config/clients/profiles/<profile>.yaml`, adding the canonical name to `allowed_tools`.
|
||||
4. Run `make validate-schemas` — it validates the new YAML against its Pydantic model before
|
||||
anything else.
|
||||
5. Run `make export-schemas` if the Pydantic model itself changed (not necessary just to add an
|
||||
example YAML to an already-existing schema).
|
||||
6. Restart the Gateway (or the Docker stack) — the Registry is seeded from
|
||||
`config/capabilities/` at boot (`registry/seed.py`); there is no hot-reload in this RI.
|
||||
|
||||
## Registering a New MCP Client
|
||||
|
||||
1. Register the client in the Keycloak realm (or OIDC equivalent) with the `client_credentials`
|
||||
grant enabled and an audience mapper for the Gateway's `OIDC_AUDIENCE` (`emcp-gateway` by
|
||||
default) — see `deploy/keycloak/realm-export.json` for the 3 example clients already
|
||||
configured this way.
|
||||
2. Create `config/clients/<client-id>.yaml` (`ClientRegistration`: `client_id`, `profile`,
|
||||
`environment`) pointing to an existing or new `ClientProfile` under
|
||||
`config/clients/profiles/`.
|
||||
3. Run `make validate-schemas`.
|
||||
4. Restart the Gateway — client registrations are also loaded at boot
|
||||
(`load_yaml_models(config.clients_dir, ClientRegistration)`).
|
||||
|
||||
## Observability
|
||||
|
||||
### Jaeger (Tracing)
|
||||
|
||||
```bash
|
||||
docker compose --profile core --profile observability up -d --build
|
||||
# UI: http://localhost:16686
|
||||
```
|
||||
|
||||
### Self-Hosted Langfuse (LLM Observability)
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f deploy/langfuse/docker-compose.override.yml \
|
||||
--profile llm-observability up -d
|
||||
# Wait for it:
|
||||
curl http://localhost:3000/api/public/health
|
||||
# Create a project/API key in the UI (http://localhost:3000) or pre-seed via
|
||||
# LANGFUSE_INIT_PROJECT_PUBLIC_KEY/LANGFUSE_INIT_PROJECT_SECRET_KEY
|
||||
# (see deploy/langfuse/docker-compose.override.yml).
|
||||
```
|
||||
|
||||
Then, point the standard OTel SDK variables to Langfuse's native OTLP endpoint (no code changes
|
||||
needed):
|
||||
|
||||
```bash
|
||||
export OTEL_EXPORTER_OTLP_ENDPOINT=http://langfuse-web:3000/api/public/otel
|
||||
export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Basic <base64(public_key:secret_key)>"
|
||||
```
|
||||
|
||||
This profile is heavy (6 containers: web/worker/postgres/clickhouse/redis/minio) and opt-in,
|
||||
decoupled from the `core`/`identity`/`observability` profiles.
|
||||
Reference in New Issue
Block a user