Commit Graph

121 Commits

Author SHA1 Message Date
nogueiraguh
2a56712665 fix: sqlite3.Row.get() compatibility in _resolve_embed_config 2026-03-11 02:36:07 -03:00
nogueiraguh
4a802c8e64 feat: knowledge base with URL import, ADB case-sensitivity fix, and RAG enhancements
- Add Knowledge Base (Base de Conhecimento) with file upload + URL import
- URL content extraction: HTML stripping, remote PDF parsing
- Auto-resolve embedding config from OCI credentials
- Case-insensitive ADB table matching with quoted identifiers
- Table validation endpoint, report_date metadata, purge & re-embed flow
- Embeddings tab cleanup, PyPDF2 dependency, README updated to v2.3
2026-03-11 02:02:31 -03:00
nogueiraguh
851def30b6 feat: Terraform intelligence, anti-flickering, and v2.2 release
- Official resource docs injection from registry.terraform.io (on-demand fetch + SQLite cache)
- Smart tfvars auto-generation from declared variables (tenancy_ocid, user_ocid, etc.)
- Multi-region provider: use var.region instead of hardcoded OCI config region
- RPC anti-cycle rule in system prompt
- System prompt auto-sync to DB on restart
- Fix UI flickering: animations scoped to tab switch only (tab-enter class)
- Terraform polling timeout increased to 60 min
- Terminal output updates without full page re-render
- Fix ${IC.bot} literal rendering in chat empty state
- Chat empty state icon scaled to 48px
- README updated to v2.2 with all changes
2026-03-10 14:29:38 -03:00
nogueiraguh
0877744cd1 docs: update README with UI modernization, fix chat icon rendering
- Update README: KPI dashboard, SVG icon system, Chart.js, report summary endpoint, ADB wallet fix, updated file sizes
- Fix ${IC.bot} rendering literally in chat empty state (single quotes → template literal)
- Scale up chat empty state SVG icon to 48px
2026-03-10 09:42:58 -03:00
nogueiraguh
4fc458cdcb feat: UI modernization phases 1B-5, KPI dashboard, charts, and ADB fix
- Phase 1B: replace all emojis with inline SVG icons (Lucide-based, 45+ icons)
- Phase 2: KPI cards for compliance reports (score, passed, failed, total)
- Phase 3: Chart.js integration (donut distribution + horizontal bar by section)
- Phase 4: SVG gauge/speedometer for compliance score with gradient arc
- Phase 5: polish animations (fade-in stagger, smooth theme transition, hover effects, skeleton loading, alert slide-in, button press, dropdown animation)
- New endpoint GET /api/reports/{rid}/summary for KPI data extraction
- Fix ADB wallet connection (DPY-6005) by always passing wallet_password
- Remove CIS_EMBEDDINGS default from ADB config
- Custom brain icon (Lucide) for GenAI section
2026-03-10 00:08:28 -03:00
nogueiraguh
f205f819ab feat: light/dark theme toggle, export/import config, and UI modernization
- Add light/dark mode toggle with localStorage persistence
- Oracle-branded dark mode: deep backgrounds, vibrant status colors
- Original light mode preserved as default
- All hardcoded colors replaced with CSS custom properties
- Add full config export/import (OCI creds+keys, GenAI, MCP, prompts, settings)
- Add Terraform inline prompt editor with auto-save
- Add ADB Update Network Access and ACL display in Explorer
- Add auto-refresh after Start/Stop resource actions
- Update README with new features
2026-03-09 15:43:32 -03:00
nogueiraguh
188fc82858 feat: export/import configuration and multiple bug fixes
- Add export/import endpoints for full config backup (OCI, GenAI, MCP, prompts, settings, ADB)
- Export includes private keys (base64) for complete environment migration
- Import merges by ID (skips existing, no duplicates)
- Fix Resource Plan parser to recognize free-form format (model skips ```plan blocks)
- Fix replan not clearing previous apply/destroy outputs
- Fix duplicate CIS Compliance Scanner MCP server on multi-worker startup
- Add Terraform prompt editor with auto-save in menu
2026-03-09 15:26:13 -03:00
nogueiraguh
c4a9f8d6d5 feat: ADB network access, prompt editor, resize persistence, and multiple fixes
- Add Update Network Access endpoint and UI for ADB ACL management
- Add inline prompt editor with auto-save in Terraform agent menu
- Persist resize bar positions across re-renders (Explorer + Terraform)
- Restore full Terraform session state from history (files, plan, outputs, OCI config)
- Parse free-form Resource Plan format (model often skips ```plan blocks)
- Auto-refresh resource state after Start/Stop actions
- Clear previous apply/destroy outputs on Re-plan
- Deduplicate CIS Compliance Scanner MCP server on startup
- Show ACL IPs in Explorer for Autonomous Databases
- Detect user IP via /api/oci/my-ip endpoint
2026-03-09 15:01:26 -03:00
nogueiraguh
01fe037cd0 docs: update README with auto-split pipeline, resource validation, and simplified prompt 2026-03-09 01:26:31 -03:00
nogueiraguh
33d1e2ddbf refactor: simplify Terraform agent system prompt 2026-03-09 01:25:32 -03:00
nogueiraguh
a5b8f6903a feat: resource type validation, 100K max tokens, and auto-split improvements
- Add SQLite-based resource type validation with suggestions for invalid types
- Increase Terraform max_tokens from 32K to 100K for large infra generation
- Generate resource reference at startup to survive volume mount overwrites
- Add invalid resource type examples to system prompt (RPC peer, subnet route table)
- Add prompt size logging for Terraform agent
- Translate remaining Portuguese code comment to English
2026-03-09 01:10:33 -03:00
nogueiraguh
6fb2eecddf feat: auto-split monolithic Terraform files and fix single-line HCL blocks
- Add 3-layer protection pipeline for model-generated Terraform code:
  1. Frontend auto-split: splits monolithic HCL into multiple files by resource type
  2. Backend auto-split: Python port as last defense before writing to disk
  3. Backend deduplication: removes monolithic leftovers with duplicate resources
- Fix single-line block expansion to handle ALL HCL blocks (nested included):
  ingress_security_rules, egress_security_rules, route_rules, match_criteria, etc.
- Add prompt size logging for terraform agent diagnostics
- Strengthen system prompt: minimum 3 files, final reminder section
2026-03-09 00:40:48 -03:00
nogueiraguh
15a8366f9c fix: delete terraform workspace and files when deleting chat session 2026-03-08 19:09:28 -03:00
nogueiraguh
c2effa1408 fix: include private_key_password in provider.tf for encrypted OCI keys 2026-03-08 19:06:03 -03:00
nogueiraguh
28a8fe4511 fix: add explicit security rules multi-line example to TF validation checklist 2026-03-08 19:03:27 -03:00
nogueiraguh
6065ba7b12 docs: remove external tool reference from README 2026-03-08 19:02:20 -03:00
nogueiraguh
1b726b1a51 fix: clarify variable region requirement in TF validation checklist 2026-03-08 19:01:05 -03:00
nogueiraguh
7f64b043d6 docs: update README with smart file correction, validation checklist, and Explorer actions 2026-03-08 02:10:43 -03:00
nogueiraguh
faeaed6181 fix: add HCL multi-line block syntax rule to TF prompt
Prevent single-line block syntax with multiple arguments which causes
terraform parse errors. All blocks must use multi-line format.
2026-03-08 01:55:39 -03:00
nogueiraguh
3c4964b1d0 fix: add region variable and peering dependency rules to TF prompt
Always declare variable "region", remove invalid resource types with
correct alternatives, and ensure RPC dependencies exist on both sides
before peering.
2026-03-08 01:51:03 -03:00
nogueiraguh
64a882d6d3 fix: add route table, DRG attachment, and RPC validation rules to TF prompt
Ensure subnets always reference valid route tables, DRG attachments
point to existing resources, and RPC peering has correct peer_id
and peer_region_name configuration.
2026-03-08 01:49:47 -03:00
nogueiraguh
34678e59ae fix: add mandatory validation checklist to Terraform Agent system prompt
Cross-references, CIDRs, security lists, route tables, dependencies,
resource type names, valid arguments, and variable declarations must
all be verified before the model delivers any code.
2026-03-08 01:48:07 -03:00
nogueiraguh
fe4eb95c46 fix: merge corrected TF files instead of replacing all
- fmTf now merges new code blocks with existing S.tfFiles: updated files
  are replaced by name, untouched files are preserved, new files are added
- System prompt changed: model generates ONLY the files that need fixing
  instead of all 25 files, drastically reducing output size
- Prevents model from discarding existing work when correcting errors
2026-03-08 01:41:18 -03:00
nogueiraguh
bb25c3dc29 feat: robust TF file correction, move Start/Stop to Explorer, increase context
- Inject existing workspace files + plan/apply output into Terraform chat
  context so model corrects in-place instead of regenerating from scratch
- Add "REGRA CRÍTICA: Correção de Arquivos Existentes" to system prompt
  requiring the model to preserve file structure and fix surgically
- Increase Terraform Agent max_tokens from 16K to 32K
- Remove "Nova conversa" button from Terraform toolbar
- Move Start/Stop Instance and ADB buttons from Terraform Resources
  panel to OCI Account Explorer (instances and databases tabs)
- Remove unused _ociResOid helper and old action functions
2026-03-08 01:34:17 -03:00
nogueiraguh
9de4c2bc7b fix: default Terraform Agent to generate multiple organized .tf files
Change the system prompt from "single file by default" to "multiple files
by default" organized by function (variables.tf, networking.tf, compute.tf,
etc.), which was causing the model to only generate 1-2 files.
2026-03-08 01:10:46 -03:00
nogueiraguh
805286c05f feat: add TF reference refresh button to Terraform Agent menu
- Add dropdown menu (⋮) in Terraform toolbar with "Atualizar Referência TF" button
- Remove standalone reference-status API endpoint, keep only refresh-reference
- Button triggers `terraform providers schema -json` regeneration with progress feedback
- Shows line count and update timestamp after successful refresh
- Update README with new line counts
2026-03-08 01:05:18 -03:00
nogueiraguh
19e481275d feat: Terraform resource reference from provider schema, provider alias fixes, and timestamps
- Add gen_tf_reference.py to generate OCI Terraform provider resource catalog (~937 resources) from `terraform providers schema -json`
- Auto-inject categorized resource reference into Terraform Agent system prompt to prevent invalid resource type names
- Build-time reference generation in Dockerfile, refreshable via POST /api/terraform/refresh-reference
- Provider alias detection with brace-matching parser for multi-region deployments
- Clean old .tf files before writing new ones on Re-plan
- Message timestamps in terraform chat and history
- Partial DOM rendering to eliminate screen flickering
- Layout overflow fix (no scroll beyond viewport)
2026-03-08 01:00:47 -03:00
nogueiraguh
8bccf9367e feat: chat history, inline file editor, split terminal, OCI resource actions, and layout fixes
- Add ChatGPT-style chat history for Chat Agent and Terraform Agent (sessions, rename, delete)
- Add inline .tf file editor with Tab support and auto-sync to workspace code
- Add split-panel terminal showing Plan/Apply/Destroy output alongside files/plan/resources
- Add OCI resource actions: start/stop Compute instances and Autonomous DBs from Resources panel
- Add multi-file Terraform generation via // filename: markers
- Fix GenAI empty response handling (GPT-5.2 fallback message)
- Fix compartment select visibility, Terraform code block light theme
- Fix page layout overflow (no more scroll beyond viewport)
- Add DB migration for terraform_workspaces.compartment_id column
- Update README with new features, API endpoints, and version notes
2026-03-08 00:15:10 -03:00
nogueiraguh
a05956a726 ui: modernize Downloads file list with compact rows, file-type icons, and hover states
Replace button-based file layout with a clean list view featuring colored
file-type badges (HTML/CSV/JSON/XLSX), formatted sizes, download icons,
hover highlights, and category headers with emoji icons. Collapsible
report cards now show file count and use chevron rotation animation.
2026-03-07 18:32:15 -03:00
nogueiraguh
c7c4ca1f1b feat: Terraform Agent, expanded OCI Explorer (40+ resources), and MCP improvements
Add Terraform Agent tab with AI-powered IaC generation (plan/apply/destroy lifecycle,
workspace management, compartment selection, Terraform CLI v1.7.5 in container).
Expand OCI Explorer from 6 to 40+ resource types across 8 categories with tree-view
navigation and resizable panels. Add compartment filtering and error tracking to MCP
CIS server, increase tool timeout to 30min with auto-retry. Add chat sidebar, chat
audit logs, and tune memory compaction (6K threshold, 20 recent messages). Trim model
catalog from 69 to 15 curated models. Update README for v2.1.
2026-03-07 12:39:20 -03:00
nogueiraguh
3e13cf4677 refactor: trim GenAI models from 69 to 15, set GPT-4.1 as default
Keep only the best model per provider family (Meta, Google, OpenAI, xAI).
Remove Codex variants, Image/Audio/Search models, ProtectAI, and redundant
versions. GPT-4.1 is now pre-selected when creating a new GenAI config.
2026-03-06 11:24:22 -03:00
nogueiraguh
2d024d3130 feat: async background chat, performance scaling, and dead code cleanup
Async chat processing eliminates 504 timeouts - POST returns immediately,
backend processes in background, frontend polls for results with timestamps.
Scale to ~12 simultaneous chats via 8 uvicorn workers + 16-thread executor.
Parallelized MCP data collection, 2h session cache, 5min tool timeout.
Full dead code cleanup across backend, frontend, MCP, docker, and nginx.
2026-03-06 11:06:21 -03:00
nogueiraguh
ef43eaa7ba feat: multimodal chat, region-specific MCP scanning, and resilience improvements
Add image/PDF/text file upload in chat via new /api/chat/upload endpoint
with OCI GenAI ImageContent and DocumentContent support. Frontend shows
attach button, file preview, and handles multipart upload seamlessly.

Add optional regions parameter to all CIS MCP scan tools for targeting
specific OCI regions instead of full tenancy. Session cache scoped by
config+regions for concurrent multi-region scans.

Add orphaned report auto-detection on progress poll (marks as failed if
process not found). Increase nginx proxy timeout to 15 minutes. Improve
$api error handling for non-JSON responses (504 HTML pages).

Update README to v1.9.
2026-03-05 13:27:17 -03:00
nogueiraguh
1d52234b1a refactor: reorganize Reports and Downloads tabs, update README
Move report history/status table to Reports tab. Simplify Downloads tab
to show only completed reports with expandable file browser grouped by
category. Add CIS Level description tooltip and fix log expand persisting
during report generation polling.
2026-03-05 11:55:05 -03:00
nogueiraguh
3cef21b6dd feat: add CIS Level description tooltip in Reports form
Show dynamic informational text below the CIS Level selector explaining
what Level 1 and Level 2 mean, updating as the user changes selection.
2026-03-05 11:29:04 -03:00
nogueiraguh
973ff65989 feat: Oracle CIS report engine, CIS engine auto-update, and granular report params
Replace lightweight cis_runner.py with Oracle's official cis_reports.py engine
(6660 lines, 48 CIS + 11 OBP checks). Add granular execution parameters (Level,
OBP, Raw Data, Redact), per-report file storage with category browser, tenancy
filter in Downloads, and individual file download.

Add CIS Engine auto-update: check/download latest cis_reports.py from Oracle's
GitHub repo with automatic patch reapplication (admin UI card + 3 new endpoints).
Save engine version to app_settings on startup.

Update README to v1.8 with new features, API endpoints, and versioning table.
2026-03-05 11:11:31 -03:00
nogueiraguh
5038403d08 feat: granular CIS MCP server, chat memory compaction, and UX improvements
- Rewrite mcp_cis_server.py with per-section scan tools (IAM, Networking,
  Compute, Logging/Monitoring, Storage, Asset Management) instead of
  monolithic full-tenancy scan — 12 granular tools
- Add cis_reports.py (Oracle CIS Benchmark checker) to backend
- Add chat memory compaction: auto-summarize old messages when history
  exceeds ~8000 tokens, keeping 6 recent messages intact
- Fix GenAI tool use loop: accumulate assistant+tool messages across
  iterations for proper conversation flow (Generic format)
- Remove tenancy confirmation section from system prompt
- Add thinking indicator and button disable in chat UI while waiting
- Add requests dependency for cis_reports.py
- Fix NoneType error in cis_reports.py region filtering
2026-03-05 08:26:43 -03:00
nogueiraguh
f259239a17 docs: update README for v1.5 with MCP Tool Use and multi-table ADB features 2026-03-04 23:24:13 -03:00
nogueiraguh
0f5c805601 feat: integrate MCP servers as tools in Chat Agent with function calling
Add MCP tool discovery (auto + manual), tool execution via MCP SDK,
and GenAI function calling loop (Cohere + Generic formats) so the
chat agent can invoke MCP server tools during conversations.

- Add mcp SDK dependency
- New endpoints: discover-tools, update tools
- Modify _call_genai to support tools/tool_results and return (text, tool_calls)
- Tool use loop in chat endpoint (max 5 iterations)
- Frontend: tools management in MCP cards, chat toggle, tools_used display
- Multi-table ADB vector search, enriched embeddings, preview chunks
- Orphaned report cleanup on container restart
- Nginx no-cache headers, searchable dropdowns, editable vector tables
2026-03-04 23:15:27 -03:00
nogueiraguh
7448f98ce0 fix: remove dated OpenAI model variants and GPT-5.2 Pro from catalog
Remove 18 duplicate OpenAI models with date suffixes (e.g. 2025-04-14)
and GPT-5.2 Pro (Responses API only) to keep the model list clean.
2026-03-04 14:58:39 -03:00
nogueiraguh
2071fa037a fix: remove unavailable Cohere models and fix xAI Grok param support
- Remove 5 Cohere chat models (not available in us-ashburn-1)
- Remove 9 legacy Cohere embedding models (v3.0), keep embed-v4.0
- Update default embedding model to cohere.embed-v4.0
- Skip presence_penalty, frequency_penalty, top_k for xAI models
- Remove Cohere from chat provider order in frontend
2026-03-04 13:53:57 -03:00
nogueiraguh
904d5c4cd7 fix: filter GPT-5.2 Pro from chat (Responses API only, not chat)
GPT-5.2 Pro and its dated variant only support the Responses API,
not Chat Completions. OCI GenAI uses chat/completions internally,
causing a 400 error. Also trim debug log to a shorter format.
2026-03-04 07:54:56 -03:00
nogueiraguh
84f35a50fe feat: replace model select with searchable dropdown in Chat tab
Custom dropdown with search filter and max-height scroll (220px).
Models grouped by provider with sticky group headers. Click outside
to close. Handles 60+ models without overwhelming the UI.
2026-03-04 07:38:31 -03:00
nogueiraguh
4de5852430 fix: filter non-chat models (Codex, Image, Audio, GPT-oss) from Chat dropdown
These models use the completions endpoint, not chat/completions,
and return 400 errors when used via the chat API.
2026-03-03 23:43:29 -03:00
nogueiraguh
749875a3cd refactor: move GenAI parameters to Chat tab, simplify GenAI Config
Remove temperature, max_tokens, top_p, top_k, frequency/presence
penalty fields from the GenAI Config tab. That tab is now focused
on adding custom models not in the catalog.

Parameters are now managed exclusively in the Chat tab via the
collapsible gear panel, available for both direct models and saved
configs. Inline params override stored defaults when using presets.
2026-03-03 20:22:47 -03:00
nogueiraguh
13b98209d5 feat: direct model selection in Chat tab without pre-configuration
Users can now pick any model from the full catalog directly in the Chat
tab using optgroups by provider (OpenAI, Google, Meta, Cohere, xAI),
without needing to pre-configure it in GenAI Config. OCI credentials,
region, and compartment auto-fill from the selected credential.

A collapsible parameters panel (temperature, max_tokens, top_p, top_k,
frequency/presence penalty) is available via the gear icon.

Saved GenAI configs still appear at the top of the dropdown as presets.
2026-03-03 20:12:26 -03:00
nogueiraguh
04e35dc681 fix: use max_completion_tokens for OpenAI models on OCI GenAI
OpenAI models reject 'max_tokens' and require 'max_completion_tokens'.
Also skip 'top_k' for OpenAI as it is unsupported. Detection is based
on the provider field in the model catalog.
2026-03-03 20:02:22 -03:00
nogueiraguh
dcd2685671 feat: add OCID resolution for all GenAI models (fixes 404 on OpenAI)
Models like OpenAI GPT-5.2 require the full OCID rather than the short
model_id. The catalog now maps model IDs to OCIDs per region (us-ashburn-1).
_call_genai and _embed_text resolve the OCID for the configured region
at runtime, falling back to short model_id for unmapped regions.

New models added from OCI portal: GPT-5.3/5.2/5.1/5 Codex, GPT Image
1/1.5, Llama Guard 4, ProtectAI DeBERTa, OpenAI Text Embedding 3
Large/Small. Total: 69 chat + 11 embedding models.
2026-03-03 19:50:55 -03:00
nogueiraguh
408ca45ee3 feat: expand GenAI catalog to 63 chat models from OCI portal
Add all models available in OCI GenAI Ashburn region including
OpenAI GPT-5.2/5.1/5/4.1/4o, o1/o3/o4-mini, GPT Audio,
xAI Grok 4.1/4 reasoning/non-reasoning variants, and update
README with complete model table and Model IDs.
2026-03-03 19:28:54 -03:00
nogueiraguh
ea71dff699 feat: improve custom model OCID support for GenAI config
- Add "Personalizado (usar OCID)" option in model select dropdown
- Validate OCID is required when custom model is selected
- Improve Model OCID field hint text explaining portal usage
- Show "OCID Personalizado" label in config table for custom models
- Allows using any OCI GenAI model via OCID from the portal
2026-03-03 19:23:50 -03:00