docs: update README for v1.2 with RAG and Embeddings features

- Update version badge to 1.2 - Add RAG description to AI Chat Agent section - Add Embeddings Management section with feature details - Add RAG Pipeline to architecture diagram - Add Embeddings API endpoints to API Reference - Add Embedding Models table (Cohere Embed v3.0/light) - Update ADB Vector setup guide with GenAI config and embedding model - Add Step 5 for Embeddings workflow - Update project structure line counts - Add v1.2 to version history
2026-03-02 20:13:01 -03:00
parent b7991d7200
commit 77a4824907
1 changed files with 56 additions and 10 deletions
--- a/README.md
+++ b/README.md
@@ -9,7 +9,7 @@
 </p>

 <p align="center">
-  <img src="https://img.shields.io/badge/version-1.1-C74634?style=flat-square" alt="Version">
+  <img src="https://img.shields.io/badge/version-1.2-C74634?style=flat-square" alt="Version">
  <img src="https://img.shields.io/badge/python-3.12-3776AB?style=flat-square" alt="Python">
  <img src="https://img.shields.io/badge/FastAPI-0.115-009688?style=flat-square" alt="FastAPI">
  <img src="https://img.shields.io/badge/OCI-GenAI-C74634?style=flat-square" alt="OCI">
@@ -23,14 +23,15 @@

 OCI CIS AI Agent is a self-hosted web application that automates **CIS Oracle Cloud Infrastructure Foundations Benchmark 3.0** compliance checks, powered by **OCI Generative AI** for intelligent analysis and an **MCP (Model Context Protocol)** server architecture for extensible task execution.

-The platform combines security compliance scanning, AI-powered chat, infrastructure exploration, and vector-based knowledge storage into a single, containerized solution with Oracle Cloud's official light theme.
+The platform combines security compliance scanning, AI-powered chat with **RAG (Retrieval-Augmented Generation)**, infrastructure exploration, and vector-based knowledge storage into a single, containerized solution with Oracle Cloud's official light theme.

 ---

 ## Features

-### 🤖 AI Chat Agent
+### 🤖 AI Chat Agent with RAG
 - **OCI Generative AI** integration via official SDK (`oci.generative_ai_inference`)
+- **RAG (Retrieval-Augmented Generation)**: automatically queries ADB vector store for relevant context before generating responses
 - 12 models across 4 providers: **Cohere** (Command A/R/R+), **Meta** (Llama 4/3.3/3.2/3.1), **Google** (Gemini 2.5), **xAI** (Grok 3/4)
 - 16 OCI regions supported with auto-generated endpoints
 - Full parameter control: temperature, max_tokens, top_p, top_k, frequency/presence penalty
@@ -63,6 +64,15 @@ The platform combines security compliance scanning, AI-powered chat, infrastruct
 - Wallet ZIP upload and automatic extraction
 - Connection testing
 - Configurable embeddings table name (default: `CIS_EMBEDDINGS`)
+- Link to GenAI config for embedding generation via OCI GenAI
+
+### 🧬 Embeddings Management
+- Dedicated tab for managing vector embeddings
+- **Embed CIS Reports**: automatically chunk reports by section (IAM, Networking, Compute, etc.) and generate embeddings
+- **Upload text files**: upload `.txt` files, automatically chunked by paragraphs
+- **OCI GenAI Embeddings**: uses Cohere Embed models (v3.0, multilingual, light) via OCI GenAI `embed_text` API
+- Browse, inspect and delete individual embeddings from the ADB vector store
+- 4 embedding models supported: Cohere Embed English/Multilingual v3.0 (1024d), Light variants (384d)

 ### 🔐 Security
 - **JWT authentication** with configurable expiry
@@ -92,6 +102,9 @@ The platform combines security compliance scanning, AI-powered chat, infrastruct
 │                          │  │ MCP Servers  │──▶ Tools    │    │
 │                          │  ├─────────────┤              │    │
 │                          │  │ oracledb     │──▶ ADB      │    │
+│                          │  ├─────────────┤              │    │
+│                          │  │ RAG Pipeline │──▶ Embed +  │    │
+│                          │  │              │   Search    │    │
 │                          │  └─────────────┘              │    │
 │                          │                                │    │
 │                          │  SQLite (agent.db)             │    │
@@ -208,14 +221,27 @@ Register MCP servers for extended task execution:

 MCP servers can be **linked to ADB Vector** databases, enabling them to use the vector store as a tool during report execution.

-### Step 4 — ADB Vector (Optional)
+### Step 4 — ADB Vector + RAG (Optional)

-For persistent vector storage of CIS findings and embeddings:
+For persistent vector storage and RAG-powered chat:

 1. Add DSN (TNS name from tnsnames.ora, e.g., `myatp_high`)
 2. Set credentials (username/password)
-3. Upload Wallet ZIP (for mTLS)
-4. Test the connection
+3. Select a **GenAI Config** (for embedding generation via OCI GenAI)
+4. Select an **Embedding Model** (Cohere Embed v3.0 recommended)
+5. Upload Wallet ZIP (for mTLS)
+6. Test the connection
+7. Click **Create Table** to initialize the embeddings table in ADB
+
+### Step 5 — Embeddings (Optional)
+
+Navigate to the **Embeddings** tab to populate the vector store:
+
+1. **From CIS Reports**: Select a completed report and generate embeddings (1 per section)
+2. **From text files**: Upload `.txt` files for automatic chunking and embedding
+3. Browse and manage existing embeddings
+
+Once embeddings exist, the **chat automatically uses RAG** — it queries the ADB vector store for relevant context before generating responses with the selected GenAI model.

 ---

@@ -241,12 +267,12 @@ Allow group <group-name> to read buckets in compartment <compartment-name>
 ```
 oci-cis-agent/
 ├── backend/
-│   ├── app.py              # FastAPI application (933 lines)
+│   ├── app.py              # FastAPI application (~1100 lines)
 │   ├── cis_runner.py        # CIS Benchmark check executor
 │   ├── Dockerfile           # Python 3.12 + OCI CLI
 │   └── requirements.txt     # Dependencies
 ├── frontend/
-│   └── index.html           # SPA with Oracle Cloud theme (419 lines)
+│   └── index.html           # SPA with Oracle Cloud theme (~620 lines)
 ├── nginx/
 │   └── default.conf         # Reverse proxy config
 ├── docker-compose.yml       # Orchestration
@@ -314,12 +340,22 @@ oci-cis-agent/

 | Method | Endpoint | Description |
 |--------|----------|-------------|
-| POST | `/api/adb/config` | Save ADB connection |
+| POST | `/api/adb/config` | Save ADB connection (with GenAI config + embedding model) |
 | GET | `/api/adb/configs` | List ADB connections |
 | POST | `/api/adb/{id}/upload-wallet` | Upload wallet ZIP |
 | POST | `/api/adb/test/{id}` | Test ADB connection |
+| POST | `/api/adb/{id}/ensure-table` | Create embeddings table in ADB |
 | DELETE | `/api/adb/configs/{id}` | Delete ADB config |

+### Embeddings
+
+| Method | Endpoint | Description |
+|--------|----------|-------------|
+| POST | `/api/embeddings/report/{rid}` | Generate embeddings from CIS report (chunked by section) |
+| POST | `/api/embeddings/upload` | Upload .txt file and generate embeddings (chunked by paragraphs) |
+| GET | `/api/embeddings/{vid}/list` | List embeddings in ADB (paginated) |
+| DELETE | `/api/embeddings/{vid}/{doc_id}` | Delete individual embedding |
+
 ### Chat & Reports

 | Method | Endpoint | Description |
@@ -360,6 +396,15 @@ oci-cis-agent/
 | xAI | Grok 4 | GENERIC |
 | xAI | Grok 3 | GENERIC |

+### Embedding Models
+
+| Provider | Model | Dimensions |
+|----------|-------|------------|
+| Cohere | Embed English v3.0 | 1024 |
+| Cohere | Embed Multilingual v3.0 | 1024 |
+| Cohere | Embed English Light v3.0 | 384 |
+| Cohere | Embed Multilingual Light v3.0 | 384 |
+
 ### GenAI Regions

 `us-chicago-1` · `us-ashburn-1` · `us-phoenix-1` · `uk-london-1` · `eu-frankfurt-1` · `ap-tokyo-1` · `ap-osaka-1` · `sa-saopaulo-1` · `ca-toronto-1` · `ap-melbourne-1` · `ap-mumbai-1` · `eu-amsterdam-1` · `me-jeddah-1` · `ap-singapore-1` · `ap-seoul-1` · `sa-vinhedo-1`
@@ -386,6 +431,7 @@ oci-cis-agent/

 | Version | Date | Changes |
 |---------|------|---------|
+| **v1.2** | 2026-03 | RAG pipeline (OCI GenAI embeddings + ADB vector search), dedicated Embeddings tab, CIS report chunking, file upload embedding |
 | **v1.1** | 2025-02 | OCI SDK GenAI (exact pattern), OCI Account Explorer, MCP↔ADB linking, full chat parameters |
 | **v1.0** | 2025-02 | Initial release: OCI theme, GenAI integration, MCP servers, ADB vector, JWT+MFA+RBAC |