Root cause observed in a real Codex session: the agent burned ~3min
on `make venv` + 4 sequential `make diagram-lookup` queries trying
to find a non-existent ref-arch (OCI PostgreSQL accessed FROM GCP
via Cross-Cloud Interconnect). The catalog has 123 entries; if two
honest queries can't surface a topology, none exists — but nothing
in the skill said "stop." Three coordinated fixes:
1) Makefile: `make venv` is now idempotent.
Marker file `.venv/.deps-installed` keyed off `requirements.txt`
mtime — the second `make venv` in a row is a 26ms no-op (was
1m53s on a Codex sandbox, paid every turn). Force a rebuild with
`rm .venv/.deps-installed`.
2) kb/architecture-center/catalog.yaml: new `known_gaps` block.
Surfaced by the lookup tool as a top-of-output banner whenever
the (synonym-expanded) query tokens match any one of a gap's
`triggers` token sets. Two gaps seeded from real engagements:
- gcp-to-oci-native-services — OCI PostgreSQL/OKE/Cache/etc.
reached FROM GCP via Cross-Cloud Interconnect (the
Database@Google Cloud entries are the OPPOSITE direction).
- newer-oci-services-without-icon — OCI Cache (Redis/Valkey)
and OCI PostgreSQL post-v24.2-toolkit; no ref-arch, no
toolkit icon, fall back to generic stencil + explicit label.
Each gap's `notice:` is a copy-paste recommended composition so
the agent doesn't have to invent a strategy.
3) tools/archcenter_pattern_lookup.py: detects + surfaces gaps.
`lookup()` now returns `{"matches": ..., "gaps": ...}`. The
text printer emits a "⚠ KNOWN GAP — <id>" banner with the
notice block ABOVE the score-based top-K, so the agent sees
the stop-and-compose signal before ever scrolling to results.
YAML output mode mirrors the same shape under `gaps:`.
4) SKILL.md (option 2 step 1) + Codex copy via sync-skill.
New explicit rule: "Lookup budget: max 2 queries — never loop."
Documents the gap banner, the closest-3-then-ask fallback, and
the prohibition against grasping at a 3rd refinement.
Verified manually:
- GCP+postgresql query → gap fires
- "redis ha multi-az" → newer-OCI gap fires
- "exadata cross region data guard" → no false-positive gap
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the procedural hole that kept making Codex sessions fall back
to ``examples/`` for YAML scaffolding even after the SKILL forbade it.
The lookup tool returns a canonical Oracle ``.drawio`` as the geometry
source, but the agent had no YAML version of that geometry to copy —
the .drawio is XML with stencil-encoded icons. Without a scaffold,
agents reverse-engineered or fell back to ``examples/`` (which is
forbidden but happened anyway because the alternative was painful).
New tool ``tools/archcenter_drawio_to_template.py`` extracts:
• canvas dimensions
• containers (region / vcn / ad / subnet) classified by stroke /
fill style markers
• service bounding boxes with ``type: TODO_identify`` (the icon
type is encoded in stencil bytes — agent fills it from the
renderer's TYPE_TO_ICON aliases when authoring the spec)
• edge waypoints (source, target, points)
Output lands at ``kb/diagram/assets/archcenter-refs/<slug>/_template.yaml``
next to the .drawio. Lookup now surfaces it as
``cached: yaml=<path>`` so the agent has a one-step "copy this YAML
and adapt" path.
Ran across all 132 cached folders: 107 templates emitted, 24 had no
.drawio (zip didn't ship one), 1 already cached.
Other persistent guardrails added in this commit:
- SKILL.md option 2 step 3: explicit reminder that valid ``type:``
values come from ``TYPE_TO_ICON`` / ``ICON_TYPE_ALIASES`` in the
renderer source plus the icon-index JSON files. Do NOT grep
``examples/`` for ``type:`` — that's where the previous Codex
session was looking, against the rule.
- ``make archcenter-templates-refresh`` Makefile target.
- The lookup index now also indexes ``_template.yaml`` per slug.
- Auto-extracted templates committed under
``kb/diagram/assets/archcenter-refs/*/_template.yaml`` so a fresh
clone has them on first lookup (no extraction step needed).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diego asked: does the persisted ``archcenter-refs-index.json`` auto-
update when the KB is updated? Partially — mtime on the cache root
catches add/remove of subdirs (POSIX), but NOT changes to files
INSIDE existing subdirs (description.md added, .drawio replaced).
Two fixes:
1. Both KB-update tools now ``touch`` ``kb/diagram/assets/archcenter-refs``
when they finish a run that produced changes:
- ``archcenter_description_fetcher.py`` after fetched > 0
- ``archcenter_zip_downloader.py`` after downloaded / extract /
overwrite events
The touch bumps the parent mtime, which the next lookup detects
and uses to rebuild its persisted index. Closes the silent-drift
case where description.md fetches landed but the index didn't
notice.
2. ``archcenter_pattern_lookup.py --rebuild-index`` flag — escape
hatch for manual KB edits (someone replaces a cached .drawio by
hand) where neither tool ran. Removes the stale index file before
the lookup, forcing a cold rebuild.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two persistent fixes flagged from a Codex session:
1. ``archcenter_pattern_lookup`` ran in 78 s on WSL2 because:
- ``_cached_assets`` did ``iterdir`` + recursive ``rglob`` over the
whole 113-folder cache for EVERY catalog entry (123x).
- ``_patterns_for`` reloaded ``reference-patterns.yaml`` for every
match (125 reloads = ~5 s).
- Description text from disk was read for every entry, even those
that wouldn't make the top-K.
Now: one-shot scan of the cache dir cached in-memory AND persisted
to ``kb/diagram/assets/archcenter-refs-index.json`` keyed by mtime;
patterns YAML loaded once and indexed by URL; a two-pass scoring
pipeline that only reads description text + cached_assets +
visual_patterns for the top-K candidates instead of every entry.
Result: cold run 14 s (one-time index build), warm run 1.1 s.
2. The drawio renderer had no alias for ``adb_s`` / ``adb_serverless``
/ ``autonomous_database_serverless`` / ``refreshable_clone`` —
Codex's spec used ``type: adb_s`` and the icon never resolved.
Mapped them to ``autonomous_database`` (the canonical 7-cell
stencil shipped by the OCI Toolkit), with ``adb_d`` as fallback.
Same proactive expansion in the drawio side that the PPTX side
already got (oci_goldengate, dynamic_routing_gateway, atp/adw,
kms/secret, identity/iam, iac/terraform, oac/oic, kafka).
The persisted lookup index is committed so a fresh clone hits warm
performance on first ``make diagram-lookup``.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Implements points 3–8 of the post-procedure follow-up.
- Make targets: diagram-lookup, diagram-validate-spec, diagram-spec-audit,
archcenter-descriptions-refresh, archcenter-smoke. Promotes the new
tools out of "remember the CLI" into discoverable build-automation.
- .gitea/workflows/diagram-validators.yaml — runs spec validator across
every absolute_layout spec on push/PR, plus a drawio re-render smoke
test on the canonical example. Hard fail blocks merges.
- archcenter_pattern_lookup --llm-rewrite — opt-in (ANTHROPIC_API_KEY
env var, mirroring DRAWIO_EXE pattern). Rewrites natural-language
queries into canonical OCI terminology before scoring; falls back to
the original query on any error.
- diagram_spec_validator — documents the policy on legacy archcenter
reproductions: LABEL_NEAR_PARENT_EDGE stays a warning so verbatim
reconstructions don't lose pixel-fidelity from cosmetic edits.
- README — documents the asset refresh procedure (downloader +
description fetcher are idempotent, refresh quarterly).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The diagram path now follows a documented standard procedure (lookup
the closest Oracle Architecture Center reference → confirm components
→ author absolute_layout → spec validator → render → visually verify)
and ships persistent guardrails so layout regressions can't recur.
Persistent procedure changes (apply to all users, all sessions):
- tools/diagram_spec_validator.py — geometry checks (CONTAINER_TOO_THIN,
CONTAINER_PADDING_VIOLATION, LABEL_OVERFLOW_PARENT) run BEFORE either
renderer (drawio + PPTX). Catches the subnet-collapse / label-overflow
bugs that the post-render drawio validator missed.
- tools/oci_diagram_gen.py + tools/oci_pptx_diagram_gen.py — call the
spec validator before emitting any output. Adds mysql / mysql_heatwave
type aliases.
- tools/archcenter_pattern_lookup.py — scores against cached page
descriptions (not just the 1-line summary), supports --queries for
multi-fragment composition, and applies synonym expansion via
kb/architecture-center/synonyms.yaml so "LB HA cross AD" matches
"load balancer high availability availability domain".
- kb/architecture-center/synonyms.yaml — canonical synonym table
(load balancer, autonomous database, data guard, …) used by the
lookup scorer.
KB enrichment:
- tools/archcenter_description_fetcher.py + 121 cached _description.md
under kb/diagram/assets/archcenter-refs/<slug>/. Removes the runtime
dependency on docs.oracle.com when authoring specs and feeds the
pattern-lookup scorer.
- 110+ cached .drawio / .svg / .png references for offline reuse,
plus the OCI Toolkit v24.2 import (kb/diagram/assets/oci-toolkit-drawio).
Documentation:
- docs/skill/output-formats.md — new "Standard diagram-generation
procedure (MANDATORY)" + geometry rules + the new validator entry.
- SKILL.md option 2 — references the mandatory procedure.
- README.md — describes the spec validator, archcenter_pattern_lookup
and description fetcher, and updates the KB-health table.
Tooling that backs the procedure (cumulative across recent sessions):
tools/archcenter_case_runner.py, archcenter_batch_driver.py,
archcenter_zip_downloader.py, drawio_visual_validator.py,
drawio_fidelity_eval.py, harvest_drawio_icon.py, import_oci_library.py,
oci_pptx_diagram_gen.py, oci_pptx_render.py, refresh_pptx_icon_index.py.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
