From ee550b6632bfb5822e3f9226db52545b3aae92ab Mon Sep 17 00:00:00 2001 From: nogueiraguh Date: Thu, 26 Mar 2026 09:43:58 -0300 Subject: [PATCH] docs: add ADB table creation SQL and detailed descriptions to README --- README.md | 64 +++++++++++++++++++++++++++++++++++++++++++------------ 1 file changed, 50 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 46bab59..b516394 100644 --- a/README.md +++ b/README.md @@ -403,30 +403,66 @@ For persistent vector storage and RAG-powered chat: #### Required ADB Vector Tables -The following tables must be created in your Autonomous Database for the auto-embedding to work correctly. Each table must have the schema: `ID VARCHAR2(100), TEXT CLOB, EMBEDDING VECTOR, METADATA CLOB`. +> **IMPORTANT**: These tables must be created in your Autonomous Database **before** using the embedding and RAG features. Without them, the auto-embedding and historical data consultation will not work. + +All tables must use the following schema: + +```sql +CREATE TABLE ( + ID RAW(16), + TEXT CLOB, + METADATA JSON, + EMBEDDING VECTOR +); +``` + +Run the SQL below to create all 11 required tables: + +```sql +-- CIS Report tables (auto-populated via "Embed Report") +CREATE TABLE summaryreportcsvvector (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE identityandaccess (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE networking (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE computeinstances (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE loggingandmonitoring (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE objectstorage (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE storageblockvolume (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE filestorageservice (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE assetmanagement (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); + +-- Knowledge base tables (populated manually via Embeddings tab) +CREATE TABLE cisrecom (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +CREATE TABLE engineerknowledgebase (ID RAW(16), TEXT CLOB, METADATA JSON, EMBEDDING VECTOR); +``` **CIS Report Tables** — auto-populated when you click "Embed Report": | Table Name | Purpose | Source CSVs | |-----------|---------|-------------| -| `summaryreportcsvvector` | CIS report summary (compliance scores, section totals) | `cis_summary_report.csv` | -| `identityandaccess` | IAM findings (users, policies, MFA, API keys) | `cis_Identity_and_Access_Management_*.csv` | -| `networking` | Network findings (security lists, NSGs, VCNs) | `cis_Networking_*.csv` | -| `computeinstances` | Compute findings (instances, metadata, boot) | `cis_Compute_*.csv` | -| `loggingandmonitoring` | Logging findings (alarms, events, notifications) | `cis_Logging_and_Monitoring_*.csv` | -| `objectstorage` | Object Storage findings (buckets, visibility, encryption) | `cis_Storage_Object_Storage_*.csv` | -| `storageblockvolume` | Block Volume findings (encryption, CMK) | `cis_Storage_Block_Volumes_*.csv` | -| `filestorageservice` | File Storage findings (encryption, CMK) | `cis_Storage_File_Storage_Service_*.csv` | -| `assetmanagement` | Asset Management findings (compartments, tagging) | `cis_Asset_Management_*.csv` | +| `summaryreportcsvvector` | CIS report summary — compliance scores per section, total controls passed/failed | `cis_summary_report.csv` | +| `identityandaccess` | IAM findings — users, policies, MFA status, API keys, password policies | `cis_Identity_and_Access_Management_*.csv` | +| `networking` | Network findings — security lists, NSGs, VCNs, open ports, ICMP rules | `cis_Networking_*.csv` | +| `computeinstances` | Compute findings — instances, metadata service, secure boot, encryption | `cis_Compute_*.csv` | +| `loggingandmonitoring` | Logging findings — alarms, events, notifications, flow logs, Cloud Guard | `cis_Logging_and_Monitoring_*.csv` | +| `objectstorage` | Object Storage findings — bucket visibility, encryption, versioning | `cis_Storage_Object_Storage_*.csv` | +| `storageblockvolume` | Block Volume findings — encryption with CMK | `cis_Storage_Block_Volumes_*.csv` | +| `filestorageservice` | File Storage findings — encryption with CMK | `cis_Storage_File_Storage_Service_*.csv` | +| `assetmanagement` | Asset Management findings — compartment usage, resource tagging | `cis_Asset_Management_*.csv` | -**Other Tables** — populated manually or via dedicated uploads: +**Knowledge Base Tables** — populated manually via Embeddings tab: | Table Name | Purpose | How to populate | |-----------|---------|-----------------| -| `cisrecom` | CIS Benchmark recommendations and best practices | Upload CIS PDF in Embeddings tab | -| `engineerknowledgebase` | General knowledge base (blogs, docs, PDFs) | Upload files or import URLs in Embeddings tab | +| `cisrecom` | CIS Benchmark official recommendations — remediation steps, rationale, audit procedures per control | Upload CIS PDF in Embeddings tab | +| `engineerknowledgebase` | General knowledge base — blogs, documentation, PDFs, URLs for complementary context | Upload files or import URLs in Embeddings tab | -> When you click **"Embed Report"** on a completed CIS report, the system automatically maps each CSV to its corresponding table and embeds all findings with tenancy name and extract date for isolation. Progress is shown in real-time. +> **How embedding works**: When you click **"Embed Report"** on a completed CIS report, the system automatically: +> 1. Maps each CSV file to its corresponding table based on the filename +> 2. Adds CIS recommendation number, section name, tenancy, and extract date to each document +> 3. Purges old data for the same tenancy/date before inserting (prevents duplicates) +> 4. Shows real-time progress with current section and queue +> +> Each embedded document contains structured metadata (`tenancy`, `extract_date`, `cis_recommendation`) enabling precise filtered searches by the Chat Agent and Consult Embeddings. ### Step 5 — Embeddings (Optional)