diff --git a/README.md b/README.md index 4965059..21181c2 100644 --- a/README.md +++ b/README.md @@ -9,7 +9,7 @@

- Version + Version Python FastAPI OCI @@ -29,13 +29,15 @@ The platform combines security compliance scanning, AI-powered chat with **RAG ( ## Features -### 🤖 AI Chat Agent with RAG +### 🤖 AI Chat Agent with RAG + MCP Tool Use - **OCI Generative AI** integration via official SDK (`oci.generative_ai_inference`) - **RAG (Retrieval-Augmented Generation)**: automatically queries ADB vector store for relevant context before generating responses +- **MCP Tool Use (Function Calling)**: GenAI models can call tools from registered MCP servers during chat — supports both Cohere and Generic (OpenAI-style) function calling formats with automatic tool execution loop (max 5 iterations) - 69 chat models + 11 embedding models across 6 providers: **Cohere**, **Meta**, **Google**, **OpenAI** (GPT-5.3/5.2/5.1/5/4.1/4o, Codex, Image, Audio, o1/o3/o4-mini, GPT-oss), **xAI** (Grok 4.1/4/3), **ProtectAI** - OCID-based model resolution: catalog maps model IDs to OCI resource IDs per region for reliable API calls - 16 OCI regions supported with auto-generated endpoints - Full parameter control: temperature, max_tokens, top_p, top_k, frequency/presence penalty +- Toggle MCP tools on/off per chat session - Conversation history with session management - On-Demand and Dedicated serving modes @@ -52,8 +54,11 @@ The platform combines security compliance scanning, AI-powered chat with **RAG ( - Optional MCP server selection per report execution - Region filtering -### 🔌 MCP Server Registry +### 🔌 MCP Server Registry + Tool Discovery - Register multiple MCP servers (stdio, SSE, Python module) +- **Automatic tool discovery**: connect to MCP servers and discover available tools with names, descriptions, and input schemas +- **Manual tool definition**: add/edit tools with JSON Schema parameter definitions +- **Chat Agent integration**: discovered tools are automatically available as GenAI function calls during chat - Upload `.py` scripts directly to servers - **Link MCP servers to ADB Vector** databases as tools - Activate/deactivate servers @@ -64,16 +69,19 @@ The platform combines security compliance scanning, AI-powered chat with **RAG ( - `python-oracledb` Thin mode (no Oracle Client needed) - Wallet ZIP upload and automatic extraction - Connection testing -- Configurable embeddings table name (default: `CIS_EMBEDDINGS`) +- **Multiple vector tables per ADB**: register, edit, toggle active/inactive for each table +- **Multi-table RAG search**: queries all active tables across all ADB configs, merges results by cosine distance - Link to GenAI config for embedding generation via OCI GenAI ### 🧬 Embeddings Management - Dedicated tab for managing vector embeddings -- **Embed CIS Reports**: automatically chunk reports by section (IAM, Networking, Compute, etc.) and generate embeddings +- **Preview chunks before embedding**: review generated sections before creating embeddings +- **Embed CIS Reports**: automatically chunk reports by section with tenancy name, regions, and compartments enrichment - **Upload text files**: upload `.txt` files, automatically chunked by paragraphs -- **OCI GenAI Embeddings**: uses Cohere Embed models (v3.0, multilingual, light) via OCI GenAI `embed_text` API +- **Table selector**: choose which ADB vector table to store embeddings in +- **OCI GenAI Embeddings**: uses Cohere Embed models (v3.0/v4.0, multilingual, light) via OCI GenAI `embed_text` API - Browse, inspect and delete individual embeddings from the ADB vector store -- 4 embedding models supported: Cohere Embed English/Multilingual v3.0 (1024d), Light variants (384d) +- 11 embedding models supported including Cohere Embed v4.0 and OpenAI Text Embedding 3 ### 📜 Configuration Logs - **Persistent activity log** per configuration tab (OCI, GenAI, ADB, MCP) @@ -107,8 +115,10 @@ The platform combines security compliance scanning, AI-powered chat with **RAG ( │ │ │ OCI SDK │──▶ OCI APIs │ │ │ │ ├─────────────┤ │ │ │ │ │ GenAI Client │──▶ LLM │ │ -│ │ ├─────────────┤ │ │ -│ │ │ MCP Servers │──▶ Tools │ │ +│ │ ├─────────────┤ (tools) │ │ +│ │ │ MCP Client │──▶ MCP Svrs │ │ +│ │ ├─────────────┤ (discover │ │ +│ │ │ │ +execute) │ │ │ │ ├─────────────┤ │ │ │ │ │ oracledb │──▶ ADB │ │ │ │ ├─────────────┤ │ │ @@ -220,7 +230,7 @@ chat_detail.compartment_id = compartment_id ### Step 3 — MCP Servers (Optional) -Register MCP servers for extended task execution: +Register MCP servers for extended task execution and **Chat Agent tool use**: | Type | Use Case | |------|----------| @@ -230,6 +240,8 @@ Register MCP servers for extended task execution: MCP servers can be **linked to ADB Vector** databases, enabling them to use the vector store as a tool during report execution. +**Tool Discovery**: After registering a server, click **"Descobrir Tools"** to automatically discover available tools via MCP protocol. You can also add tools manually with name, description, and JSON Schema parameters. Discovered tools are automatically available as **function calls** in the Chat Agent. + ### Step 4 — ADB Vector + RAG (Optional) For persistent vector storage and RAG-powered chat: @@ -237,20 +249,21 @@ For persistent vector storage and RAG-powered chat: 1. Add DSN (TNS name from tnsnames.ora, e.g., `myatp_high`) 2. Set credentials (username/password) 3. Select a **GenAI Config** (for embedding generation via OCI GenAI) -4. Select an **Embedding Model** (Cohere Embed v3.0 recommended) +4. Select an **Embedding Model** (Cohere Embed v4.0 recommended) 5. Upload Wallet ZIP (for mTLS) 6. Test the connection -7. Click **Create Table** to initialize the embeddings table in ADB +7. **Register vector tables**: add the names of existing tables in your ADB that contain vectorized data (e.g., `CIS_REPORT`, `CIS_RECOMMENDATIONS`). Toggle tables active/inactive to control which are queried during RAG. ### Step 5 — Embeddings (Optional) Navigate to the **Embeddings** tab to populate the vector store: -1. **From CIS Reports**: Select a completed report and generate embeddings (1 per section) +1. **From CIS Reports**: Select a completed report, **preview chunks** (with tenancy/regions/compartments context), then generate embeddings 2. **From text files**: Upload `.txt` files for automatic chunking and embedding -3. Browse and manage existing embeddings +3. **Select target table**: choose which ADB vector table to store embeddings in +4. Browse and manage existing embeddings per table -Once embeddings exist, the **chat automatically uses RAG** — it queries the ADB vector store for relevant context before generating responses with the selected GenAI model. +Once embeddings exist, the **chat automatically uses RAG** — it queries all active vector tables across all ADB configs for relevant context before generating responses with the selected GenAI model. --- @@ -276,12 +289,12 @@ Allow group to read buckets in compartment ``` oci-cis-agent/ ├── backend/ -│ ├── app.py # FastAPI application (~1100 lines) +│ ├── app.py # FastAPI application (~2200 lines) │ ├── cis_runner.py # CIS Benchmark check executor │ ├── Dockerfile # Python 3.12 + OCI CLI │ └── requirements.txt # Dependencies ├── frontend/ -│ └── index.html # SPA with Oracle Cloud theme (~620 lines) +│ └── index.html # SPA with Oracle Cloud theme (~900 lines) ├── nginx/ │ └── default.conf # Reverse proxy config ├── docker-compose.yml # Orchestration @@ -346,6 +359,8 @@ oci-cis-agent/ | PUT | `/api/mcp/servers/{id}/toggle` | Activate/deactivate | | POST | `/api/mcp/servers/{id}/upload` | Upload script file | | PUT | `/api/mcp/servers/{id}/link-adb` | Link to ADB Vector | +| POST | `/api/mcp/servers/{id}/discover-tools` | Auto-discover tools from MCP server | +| PUT | `/api/mcp/servers/{id}/tools` | Manually update tool definitions | | DELETE | `/api/mcp/servers/{id}` | Delete MCP server | ### ADB Vector @@ -353,28 +368,32 @@ oci-cis-agent/ | Method | Endpoint | Description | |--------|----------|-------------| | POST | `/api/adb/config` | Save ADB connection (with GenAI config + embedding model) | -| GET | `/api/adb/configs` | List ADB connections | +| GET | `/api/adb/configs` | List ADB connections (includes vector tables) | | PUT | `/api/adb/configs/{id}` | Update ADB connection (multipart) | | POST | `/api/adb/parse-wallet` | Parse wallet ZIP and extract DSN names | | POST | `/api/adb/{id}/upload-wallet` | Upload wallet ZIP | | POST | `/api/adb/test/{id}` | Test ADB connection | -| POST | `/api/adb/{id}/ensure-table` | Create embeddings table in ADB | +| GET | `/api/adb/{id}/tables` | List vector tables for ADB config | +| POST | `/api/adb/{id}/tables` | Add vector table | +| PUT | `/api/adb/{id}/tables/{tid}` | Update vector table (name, description, active) | +| DELETE | `/api/adb/{id}/tables/{tid}` | Remove vector table | | DELETE | `/api/adb/configs/{id}` | Delete ADB config | ### Embeddings | Method | Endpoint | Description | |--------|----------|-------------| -| POST | `/api/embeddings/report/{rid}` | Generate embeddings from CIS report (chunked by section) | -| POST | `/api/embeddings/upload` | Upload .txt file and generate embeddings (chunked by paragraphs) | -| GET | `/api/embeddings/{vid}/list` | List embeddings in ADB (paginated) | -| DELETE | `/api/embeddings/{vid}/{doc_id}` | Delete individual embedding | +| GET | `/api/embeddings/preview/{rid}` | Preview report chunks before embedding (with tenancy/regions/compartments) | +| POST | `/api/embeddings/report/{rid}` | Generate embeddings from CIS report (chunked by section, accepts `table_name`) | +| POST | `/api/embeddings/upload` | Upload .txt file and generate embeddings (accepts `table_name`) | +| GET | `/api/embeddings/{vid}/list` | List embeddings in ADB (paginated, accepts `table_name` query param) | +| DELETE | `/api/embeddings/{vid}/{doc_id}` | Delete individual embedding (accepts `table_name` query param) | ### Chat & Reports | Method | Endpoint | Description | |--------|----------|-------------| -| POST | `/api/chat` | Send message (with optional GenAI model) | +| POST | `/api/chat` | Send message (with RAG + MCP tool use, accepts `use_tools` flag) | | POST | `/api/reports/run` | Execute CIS report | | GET | `/api/reports` | List reports | | GET | `/api/reports/{id}/html` | View HTML report | @@ -514,7 +533,7 @@ All models include OCID mapping for `us-ashburn-1`. For other regions, use the " | ADB | `python-oracledb` 2.4.1 (Thin mode) | | GenAI | `oci.generative_ai_inference` | | Container | Docker Compose, Nginx reverse proxy | -| MCP | Model Context Protocol (stdio/SSE/module) | +| MCP | Model Context Protocol SDK (stdio/SSE) with tool discovery + execution | --- @@ -522,6 +541,7 @@ All models include OCID mapping for `us-ashburn-1`. For other regions, use the " | Version | Date | Changes | |---------|------|---------| +| **v1.5** | 2026-03 | MCP Tool Use in Chat (GenAI function calling with auto tool discovery + execution via MCP SDK), multi-table ADB vector search, preview chunks before embedding, enriched embeddings with tenancy/regions/compartments, searchable dropdowns, editable vector tables, orphaned report cleanup on restart | | **v1.4** | 2026-03 | In-place config editing, 69 chat + 11 embedding models with OCID resolution (OpenAI GPT-5.3/5.2/5.1/5/4.1/4o/Codex/Image/Audio, xAI, Google, Meta, ProtectAI), OpenAI Text Embedding 3, custom OCID support, wallet auto-parse with DSN extraction | | **v1.3** | 2026-03 | Persistent config logs per tab, GenAI auto-fill from OCI credentials, inline UX feedback with loading spinners, MCP type-switch fix, encrypted key passphrase support, Docker stdin hang fix | | **v1.2** | 2026-03 | RAG pipeline (OCI GenAI embeddings + ADB vector search), dedicated Embeddings tab, CIS report chunking, file upload embedding |