chore: cleanup git — remove IDE files, LESSONS_LEARNED, update README

- Removed from git: .idea/, .DS_Store, LESSONS_LEARNED.md, frontend-react/README.md
- Updated .gitignore: added LESSONS_LEARNED.md, dist/, .vscode/, **/.DS_Store
- Removed legacy frontend/ directory from disk
- README: updated thread pool 16→10, Terraform 1.7.5→1.14.7, i18n 625→700+ keys
- README: added RAG optimizations (single connection, smart skip, follow-up enrichment, source hierarchy)
- README: added embedding features (auto-map, per-section, CIS tagging, progress bar, background persistence)
- README: updated Dockerfile description (Chromium), stores description (app state persistence)

README.md

@@ -31,12 +31,13 @@ The platform combines security compliance scanning, AI-powered chat with **RAG (
 
 ### 🤖 AI Chat Agent with RAG + MCP Tool Use
 - **OCI Generative AI** integration via official SDK (`oci.generative_ai_inference`)
-- **RAG (Retrieval-Augmented Generation)**: automatically queries ADB vector store for relevant context before generating responses
+- **RAG (Retrieval-Augmented Generation)**: automatically queries ADB vector store for relevant context before generating responses — single ADB connection per search, smart table skip (only queries relevant tables based on question topic), CIS recommendation text filter for exact matching, follow-up context enrichment for short questions
+- **Source hierarchy**: findings tables > cisrecom (official remediation) > engineerknowledgebase (complementary) — temporal awareness with extract dates, tenancy isolation via JSON_VALUE filter
 - **MCP Tool Use (Function Calling)**: GenAI models can call tools from registered MCP servers during chat — supports both Cohere and Generic (OpenAI-style) function calling formats with automatic tool execution loop (max 5 iterations)
 - **Chat Memory Compaction**: automatic summarization of older messages when conversation exceeds ~6000 tokens — keeps 20 recent messages (10 interactions) intact and generates an LLM-based summary of older context
 - **Chat Sidebar**: collapsible right-side panel with model parameters (temperature, max_tokens, top_p, top_k, penalties) for quick adjustment during conversation
 - **Multimodal Chat**: upload images (PNG/JPG/GIF/WebP), PDFs, and text files directly in the chat for AI analysis — supports up to 5 files per message via OCI GenAI `ImageContent` and `DocumentContent`
-- **Async Background Processing**: chat requests return immediately, GenAI + MCP tools process in background via dedicated thread pool (16 threads), frontend polls for results — eliminates 504 timeouts on long-running scans
+- **Async Background Processing**: chat requests return immediately, GenAI + MCP tools process in background via dedicated thread pool (10 threads), frontend polls for results — eliminates 504 timeouts on long-running scans
 - **Thinking Indicator**: button disables and shows spinner + "Pensando..." while waiting for GenAI response, with message timestamps
 - 16 chat models + 3 embedding models across 5 providers: **Meta** (Llama 4), **Google** (Gemini 2.5), **OpenAI** (GPT-5.2/5.1/5 Mini/4.1/4o, o3/o4-mini), **xAI** (Grok 4/3)
 - OCID-based model resolution: catalog maps model IDs to OCI resource IDs per region for reliable API calls
@@ -65,7 +66,7 @@ The platform combines security compliance scanning, AI-powered chat with **RAG (
 - **Resource type validation**: SQLite-based validation of ~937 OCI resource types with `difflib.get_close_matches` suggestions for invalid types — warnings appended automatically to model responses
 - **Chat history**: ChatGPT-style session history with rename/delete, shared between Chat Agent and Terraform Agent
 - **Resizable split view**: chat on top (50%), files/plan/resources + terminal on bottom (50%) with drag-to-resize
-- Terraform CLI installed in container (v1.7.5)
+- Terraform CLI installed in container (v1.14.7)
 - **Terraform Resource Reference**: auto-generated OCI provider resource catalog (~937 resources) injected into system prompt — built from `terraform providers schema -json` at container startup, stored in SQLite, refreshable via menu button
 - **Official Resource Docs Injection**: on-demand fetch of Example Usage + Argument Reference from the official OCI Terraform provider docs (registry.terraform.io) — cached in SQLite, injected into model context based on detected resource types
 - **Smart variable resolution**: auto-generates `terraform.tfvars` by scanning declared variables and mapping them to OCI config values (`tenancy_ocid`, `user_ocid`, `fingerprint`, `compartment_id`, `ssh_public_key`) — region is never forced, respecting variable defaults
@@ -188,17 +189,19 @@ The platform combines security compliance scanning, AI-powered chat with **RAG (
 - **Case-insensitive table matching**: supports both uppercase and lowercase table names in Oracle ADB with quoted identifiers
 
 ### 🧬 Embeddings & Knowledge Base
-- Dedicated tab for managing vector embeddings and knowledge base
-- **CIS Recommendations**: upload CIS PDF to populate the `cisrecom` vector table with Oracle Cloud security recommendations
-- **Knowledge Base (Base de Conhecimento)**: upload documents (`.txt`, `.pdf`, `.csv`, `.json`, `.md`) or **import URLs** (web pages, PDF links) to populate the `engineerknowledgebase` vector table — content is automatically extracted, chunked, and vectorized
-- **URL import**: fetches web pages, strips HTML tags/scripts, extracts readable text, and generates embeddings — supports HTML pages and remote PDFs
-- **Embed CIS Reports**: automatically chunk reports by section with tenancy name, regions, and compartments enrichment, with `report_date` metadata for tracking data freshness
-- **Purge & re-embed**: option to purge old embeddings by tenancy before re-embedding updated reports
-- **Preview chunks before embedding**: review generated sections before creating embeddings
-- **OCI GenAI Embeddings**: uses Cohere Embed models (v4.0, multilingual) via OCI GenAI `embed_text` API
-- **Consult Embeddings**: dedicated sub-menu under Embeddings with chat-like interface — query the vector store in natural language, AI searches all active tables via cosine similarity and returns contextual answers with source citations
-- Browse and inspect embeddings from the ADB vector store
-- 3 embedding models supported: Cohere Embed v4.0, OpenAI Text Embedding 3 Large/Small
+- **Auto-embed CIS Reports**: one-click embedding of all report CSVs — each CSV automatically mapped to its ADB table (identityandaccess, networking, computeinstances, etc.)
+- **Per-section embedding**: embed individual sections (Identity, Networking, Compute, etc.) with dedicated button per section
+- **CIS Recommendation tagging**: each embedded document includes CIS recommendation number, section, and status in the content for precise search
+- **Auto-detect embedding dimension**: detects table dimension (1536/3072) and selects correct model automatically
+- **Purge before re-embed**: automatically deletes old data for same tenancy/date before inserting new embeddings
+- **Real-time progress**: progress bar with current section, global progress, queue of upcoming sections
+- **Background persistence**: embedding continues when navigating away, progress resumes on return
+- **Text chunking**: large CSV rows split into 8000-char chunks with context header (Tenancy, Resource, CIS Recommendation) repeated
+- **CIS Recommendations**: upload CIS PDF to populate the `cisrecom` vector table
+- **Knowledge Base**: upload documents (`.txt`, `.pdf`, `.csv`, `.json`, `.md`) or **import URLs** to populate `engineerknowledgebase`
+- **Consult Embeddings**: chat-like interface with tenancy selector — queries vector store with CIS number detection for exact matching, smart table skip, source citations
+- **OCI GenAI Embeddings**: uses OpenAI Text Embedding 3 Large/Small via OCI GenAI
+- 11 ADB vector tables supported (9 CIS report + 2 knowledge base)
 
 ### 📜 Configuration & Chat Logs
 - **Persistent activity log** per configuration tab (OCI, GenAI, ADB, MCP)
@@ -464,15 +467,15 @@ oci-cis-agent/
 │   ├── cis_reports.py       # Oracle CIS Benchmark checker (6660 lines, report engine)
 │   ├── mcp_cis_server.py    # MCP server with 12 granular CIS tools (~700 lines)
 │   ├── gen_tf_reference.py  # OCI Terraform provider resource catalog generator
-│   ├── Dockerfile           # Python 3.12 + OCI CLI + Terraform CLI
+│   ├── Dockerfile           # Python 3.12 + OCI CLI + Terraform 1.14.7 + Chromium
 │   └── requirements.txt     # Dependencies
 ├── frontend-react/
 │   ├── src/                 # React 19 SPA (TypeScript, 38 source files, ~15500 lines)
 │   │   ├── pages/           # 15 page components (Chat, Terraform, Explorer, Reports, Config, Admin)
 │   │   ├── api/             # API client + endpoint modules
-│   │   ├── stores/          # Zustand auth store
-│   │   ├── hooks/           # Custom hooks (theme, i18n)
-│   │   └── i18n/            # Internationalization (pt/en, 625 keys)
+│   │   ├── stores/          # Zustand stores (app state persistence across navigation)
+│   │   ├── hooks/           # Custom hooks (theme, polling)
+│   │   └── i18n/            # Internationalization (pt/en, 700+ keys)
 │   └── dist/                # Built SPA served at /
 ├── nginx/
 │   └── default.conf         # Reverse proxy (React SPA + API /api/)
@@ -741,7 +744,7 @@ All models include OCID mapping for `us-ashburn-1`. For other regions, use the "
 | GenAI | `oci.generative_ai_inference` |
 | Container | Docker Compose, Nginx reverse proxy |
 | MCP | Model Context Protocol SDK (stdio/SSE) with tool discovery + execution |
-| Terraform | Terraform CLI 1.7.5, OCI Provider |
+| Terraform | Terraform CLI 1.14.7, OCI Provider |
 | CIS Scanner | Oracle CIS Foundations Benchmark 3.0 checker (`cis_reports.py`) |
 
 ---