Skill: kill the lookup-loop time-sink (3 fixes from Codex transcript)
Root cause observed in a real Codex session: the agent burned ~3min
on `make venv` + 4 sequential `make diagram-lookup` queries trying
to find a non-existent ref-arch (OCI PostgreSQL accessed FROM GCP
via Cross-Cloud Interconnect). The catalog has 123 entries; if two
honest queries can't surface a topology, none exists — but nothing
in the skill said "stop." Three coordinated fixes:
1) Makefile: `make venv` is now idempotent.
Marker file `.venv/.deps-installed` keyed off `requirements.txt`
mtime — the second `make venv` in a row is a 26ms no-op (was
1m53s on a Codex sandbox, paid every turn). Force a rebuild with
`rm .venv/.deps-installed`.
2) kb/architecture-center/catalog.yaml: new `known_gaps` block.
Surfaced by the lookup tool as a top-of-output banner whenever
the (synonym-expanded) query tokens match any one of a gap's
`triggers` token sets. Two gaps seeded from real engagements:
- gcp-to-oci-native-services — OCI PostgreSQL/OKE/Cache/etc.
reached FROM GCP via Cross-Cloud Interconnect (the
Database@Google Cloud entries are the OPPOSITE direction).
- newer-oci-services-without-icon — OCI Cache (Redis/Valkey)
and OCI PostgreSQL post-v24.2-toolkit; no ref-arch, no
toolkit icon, fall back to generic stencil + explicit label.
Each gap's `notice:` is a copy-paste recommended composition so
the agent doesn't have to invent a strategy.
3) tools/archcenter_pattern_lookup.py: detects + surfaces gaps.
`lookup()` now returns `{"matches": ..., "gaps": ...}`. The
text printer emits a "⚠ KNOWN GAP — <id>" banner with the
notice block ABOVE the score-based top-K, so the agent sees
the stop-and-compose signal before ever scrolling to results.
YAML output mode mirrors the same shape under `gaps:`.
4) SKILL.md (option 2 step 1) + Codex copy via sync-skill.
New explicit rule: "Lookup budget: max 2 queries — never loop."
Documents the gap banner, the closest-3-then-ask fallback, and
the prohibition against grasping at a 3rd refinement.
Verified manually:
- GCP+postgresql query → gap fires
- "redis ha multi-az" → newer-OCI gap fires
- "exadata cross region data guard" → no false-positive gap
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -327,12 +327,42 @@ def _patterns_for(entry: dict) -> list[str]:
|
||||
return _build_patterns_index().get(entry.get("url", ""), [])
|
||||
|
||||
|
||||
def lookup(query: str, top: int = 5, expand_synonyms: bool = True) -> list[dict]:
|
||||
def _matched_gaps(query_tokens: set[str], gaps: list[dict]) -> list[dict]:
|
||||
"""Return every known gap whose triggers match the (expanded) query.
|
||||
|
||||
A gap fires when ANY one of its `triggers` token sets is a subset of
|
||||
`query_tokens` — i.e. AND inside each set, OR across sets. This
|
||||
matches the "stop searching, compose instead" intent: if even one
|
||||
narrow trigger pair (e.g. `[gcp, postgresql]`) is in the query, the
|
||||
catalog has nothing canonical to offer.
|
||||
"""
|
||||
matched: list[dict] = []
|
||||
for gap in gaps or []:
|
||||
triggers = gap.get("triggers") or []
|
||||
for trigger_set in triggers:
|
||||
tokens_required = {t.lower() for t in trigger_set}
|
||||
if tokens_required and tokens_required.issubset(query_tokens):
|
||||
matched.append(gap)
|
||||
break
|
||||
return matched
|
||||
|
||||
|
||||
def lookup(query: str, top: int = 5, expand_synonyms: bool = True) -> dict:
|
||||
"""Score the catalog against `query` and return the top-K plus any
|
||||
known-gap notices the query triggers.
|
||||
|
||||
Returns a dict shaped as:
|
||||
{"matches": [<top-K entry dicts>], "gaps": [<matched gap dicts>]}
|
||||
|
||||
Callers that only care about matches can read `result["matches"]`.
|
||||
"""
|
||||
catalog = yaml.safe_load(CATALOG.read_text(encoding="utf-8"))
|
||||
entries = catalog.get("entries", [])
|
||||
gaps_def = catalog.get("known_gaps", []) or []
|
||||
qt = _tokens(query)
|
||||
if expand_synonyms:
|
||||
qt = _expand_query_tokens(qt)
|
||||
matched_gaps = _matched_gaps(qt, gaps_def)
|
||||
scored = []
|
||||
# Two-pass scoring: cheap fields first (title/tags/services/summary),
|
||||
# then enrich the top-K with expensive fields (description text from
|
||||
@@ -372,10 +402,19 @@ def lookup(query: str, top: int = 5, expand_synonyms: bool = True) -> list[dict]
|
||||
"visual_patterns": _patterns_for(e),
|
||||
"has_description": bool(c.get("_description")),
|
||||
})
|
||||
return scored
|
||||
return {"matches": scored, "gaps": matched_gaps}
|
||||
|
||||
|
||||
def _print_results(query: str, results: list[dict]) -> None:
|
||||
def _print_results(query: str, payload: dict) -> None:
|
||||
gaps = payload.get("gaps") or []
|
||||
results = payload.get("matches") or []
|
||||
for gap in gaps:
|
||||
notice = (gap.get("notice") or "").rstrip()
|
||||
gap_id = gap.get("id", "unknown")
|
||||
print(f"⚠ KNOWN GAP — {gap_id}")
|
||||
for line in notice.splitlines():
|
||||
print(f" {line}" if line else "")
|
||||
print()
|
||||
if not results:
|
||||
print(f"No matches for: {query}")
|
||||
return
|
||||
|
||||
Reference in New Issue
Block a user