GustavoNogueira/A-Team-Security-Infra-Agent-Engineer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: consult embeddings UI, vector search FLOAT32 fix, and base64 compartment decode

Browse Source 
- Add Consult Embeddings sub-menu with chat-like Q&A interface
- Fix VECTOR_DISTANCE FLOAT32/FLOAT64 mismatch (array 'd' → 'f')
- Decode base64 compartment_id in _resolve_embed_config
- Sidebar sub-item navigation for Embeddings hierarchy
- Fallback to raw document display when no GenAI config available
This commit is contained in:
nogueiraguh
2026-03-11 03:06:53 -03:00
parent 2a56712665
commit 749f4e479b
3 changed files with 136 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
README.md
View File

@@ -159,6 +159,7 @@ The platform combines security compliance scanning, AI-powered chat with **RAG (
- **Purge & re-embed**: option to purge old embeddings by tenancy before re-embedding updated reports
- **Preview chunks before embedding**: review generated sections before creating embeddings
- **OCI GenAI Embeddings**: uses Cohere Embed models (v4.0, multilingual) via OCI GenAI `embed_text` API
- **Consult Embeddings**: dedicated sub-menu under Embeddings with chat-like interface — query the vector store in natural language, AI searches all active tables via cosine similarity and returns contextual answers with source citations
- Browse and inspect embeddings from the ADB vector store
- 3 embedding models supported: Cohere Embed v4.0, OpenAI Text Embedding 3 Large/Small
@@ -536,6 +537,7 @@ oci-cis-agent/
| POST | `/api/embeddings/report/{rid}` | Generate embeddings from CIS report (chunked by section, accepts `table_name`, `report_date`) |
| POST | `/api/embeddings/upload` | Upload file (.txt/.pdf/.csv/.json/.md) and generate embeddings |
| POST | `/api/embeddings/upload-url` | Import URL (web page or PDF), extract text, and generate embeddings |
| POST | `/api/embeddings/consult` | Query embeddings via vector search + GenAI (natural language Q&A) |
| POST | `/api/embeddings/{vid}/purge` | Purge old embeddings by table + tenancy before re-embedding |
| GET | `/api/embeddings/{vid}/list` | List embeddings in ADB (paginated, accepts `table_name` query param) |
| DELETE | `/api/embeddings/{vid}/{doc_id}` | Delete individual embedding (accepts `table_name` query param) |
@@ -667,7 +669,7 @@ All models include OCID mapping for `us-ashburn-1`. For other regions, use the "
| Version | Date | Changes |
|---------|------|---------|
| **v2.3** | 2026-03 | **Knowledge Base & RAG Enhancements**: Knowledge Base (Base de Conhecimento) with file upload + URL import to `engineerknowledgebase` vector table, CIS Recommendations upload to `cisrecom` table, URL content extraction (HTML tag/script stripping, remote PDF parsing), auto-resolve embedding config from OCI credentials (no separate GenAI config required), `report_date` metadata tracking for embedding freshness, purge & re-embed flow for updated reports. **ADB Vector Improvements**: case-insensitive table matching with quoted identifiers for Oracle ADB, table validation endpoint against `user_tables`, tables section moved inside edit connection card, case-preserving table registration (removed forced uppercase). **Embeddings tab cleanup**: removed delete buttons, simplified to read-only view, updated descriptions. **Dependencies**: added PyPDF2 for PDF text extraction |
| **v2.3** | 2026-03 | **Consult Embeddings**: dedicated sub-menu with chat-like interface for natural language Q&A against vector store — queries all active tables via cosine similarity, returns contextual answers with source citations (works with or without GenAI config). **Knowledge Base & RAG Enhancements**: Knowledge Base (Base de Conhecimento) with file upload + URL import to `engineerknowledgebase` vector table, CIS Recommendations upload to `cisrecom` table, URL content extraction (HTML tag/script stripping, remote PDF parsing), auto-resolve embedding config from OCI credentials (no separate GenAI config required), `report_date` metadata tracking for embedding freshness, purge & re-embed flow for updated reports. **ADB Vector Improvements**: case-insensitive table matching with quoted identifiers for Oracle ADB, table validation endpoint against `user_tables`, tables section moved inside edit connection card, case-preserving table registration (removed forced uppercase). **Vector search fix**: FLOAT32 compatibility for VECTOR_DISTANCE queries, base64-decoded compartment_id for auto-resolved OCI configs. **Embeddings tab cleanup**: removed delete buttons, simplified to read-only view, updated descriptions. **Dependencies**: added PyPDF2 for PDF text extraction |
| **v2.2** | 2026-03 | **UI Modernization (Phases 1B-5)**: Oracle Dark Premium theme, 45+ Lucide SVG icons replacing all emojis, KPI dashboard with compliance gauge + Chart.js donut/bar charts, report summary API endpoint, Phase 5 animations (staggered fade-in, smooth theme transitions, hover effects) — animations scoped to tab switch only (no flickering on re-render). **Terraform Intelligence**: official resource docs injection from registry.terraform.io (Example Usage + Argument Reference, on-demand fetch + SQLite cache), smart tfvars auto-generation from declared variables, multi-region provider management with `var.region` support, RPC anti-cycle rule, system prompt auto-sync to DB, 60-min polling timeout for long-running applies. **ADB fix**: wallet_password always passed for thin mode mTLS (fixes DPY-6005). CIS_EMBEDDINGS default removed from ADB config |
| **v2.1** | 2026-03 | Terraform Agent (AI-powered IaC generation with plan/apply/destroy lifecycle, workspace management, Terraform CLI in container, inline file editor, split-panel terminal, multi-file generation via `// filename:` markers, **smart file correction** with merge-based updates, **14-point validation checklist**, 100K max_tokens), **3-layer auto-split pipeline** (frontend + backend monolithic HCL splitting into categorized files + backend deduplication), **HCL syntax auto-fix** (single-line blocks expanded to multi-line), **SQLite-based resource type validation** (~937 types with `difflib` suggestions for invalid types), **Terraform Resource Reference** (auto-generated OCI provider catalog from `terraform providers schema -json`, generated at startup to survive volume mounts, stored in SQLite, refreshable via menu button), compact system prompt for Terraform agent, ChatGPT-style chat history for both Chat Agent and Terraform Agent (rename/delete sessions), OCI Resource Actions (start/stop instances and Autonomous DBs from **OCI Account Explorer**), OCI Explorer expanded to 40+ resource types across 8 categories with tree-view navigation and resizable panels, compartment filtering for MCP CIS scans, chat sidebar with model parameters, chat audit logs, collection error tracking in MCP server, tool timeout increased to 30min with auto-retry, memory compaction tuned (6K threshold, 20 recent messages), model catalog trimmed from 69 to 15 curated models, improved GenAI response extraction with fallback for empty responses, fixed layout overflow (no page scroll), provider alias detection with brace-matching parser, multi-region Terraform support |
| **v2.0** | 2026-03 | Async background chat processing (no more 504 timeouts), frontend polling with timestamps, 8 uvicorn workers + 16-thread chat executor for ~12 simultaneous chats, parallelized MCP data collection (5-thread base + 8-thread regional), 2-hour MCP session cache, 5-min tool timeout, full dead code cleanup across backend/frontend/MCP |