nogueiraguh ab2d0a3e9b fix: embedding reliability - chunking, truncation, progress bar, shared status ...

- Text chunking: max 8000 chars per document, large CSV rows split with context header
- Text truncation: _embed_text truncates at 8000 chars before sending to model (prevents 400 errors)
- Purge before re-embed: deletes old docs by tenancy+date before inserting new ones
- Shared embedding status: file-based status (/data/.embed_status/) instead of in-memory dict (fixes multi-worker visibility)
- Progress bar: shows green (OK) + red (failures) segments, percentage, processed/total count
- Better error logging: extracts readable error message from OCI API errors

2026-03-21 10:48:21 -03:00

392 KiB

Raw Blame History

View Raw