docs: atualiza execução local, configuração e arquitetura de voz

README.md

@@ -29,7 +29,8 @@ flowchart TB
     U --> P[VoicePipeline]
     P --- S[SessionState · session and audio sequence]
     P --- T[TurnManager · semantic turn state]
-    G --> STT[NVIDIA ASR · STT per turn]
+    U --> STT[NVIDIA ASR · STT per turn]
+    U --> VAD[Silero VAD · local CPU · per continuous session]
     STT -->|Transcript events| P
     P --> A[Mock Agent · per session]
     A -->|Response text| P
@@ -219,9 +220,10 @@ flowchart LR
 
 | Component | Owns |
 |---|---|
-| Audio Gateway | WebSocket connection, protocol messages, provider cleanup coordination |
+| Audio Gateway | WebSocket connection, protocol validation, session composition, event delivery, and shielded application cleanup |
 | SessionState | Session lifecycle and audio sequence validation |
 | TurnManager | Semantic turn state transitions |
+| VoiceSession | Input history, VAD boundaries, STT lifecycle, finalization, interruption, and provider cleanup |
 | VoicePipeline | Turn execution task and Agent/TTS orchestration |
 | OCI streaming bridge | SSE reader, event queue, and stream closure |
 | Provider adapters | Provider-specific requests, connections, and data formats |
@@ -275,6 +277,24 @@ APP_ENV=development uv run uvicorn \
   --host 127.0.0.1 --port 8001
 ```
 
+Open `http://127.0.0.1:8001/ui`, allow microphone access, and click **Iniciar conversa**.
+The client sends continuous audio; backend Silero detects turns and new speech
+interrupts an active response. Audio capture continues during playback.
+
+Transcripts currently appear in the browser developer console as `[STT PARTIAL]`
+and `[STT FINAL]`; Agent text appears as `[AGENT RESPONSE]` or `[AGENT DELTA]`.
+On Firefox for macOS, open the console with **Command + Option + K**.
+The page displays conversation status and plays synthesized audio.
+
+The Gateway and VAD run locally without LiveKit or a LiveKit Cloud account.
+STT and TTS are separate services and may run remotely; the loopback addresses
+in development must point to local services or active port forwards. Audio is
+sent to those services, so the full processing path is not offline. Model and
+Python dependencies must be installed before offline VAD execution.
+
+If port 8001 is occupied, choose another port (for example, 8002) and use that
+same port in the browser, health checks, and WebSocket tools.
+
 Check availability:
 
 ```bash
@@ -293,6 +313,7 @@ curl http://127.0.0.1:8001/ready
 | Speech-to-text | NVIDIA ASR | `127.0.0.1:9000`, `pt-BR`, punctuation enabled |
 | Agent | Mock Agent | In-process; simulated text response |
 | Text-to-speech | VoxCPM | `http://127.0.0.1:8000`, `openbmb/VoxCPM2`, `default` voice |
+| Voice activity | Silero ONNX | Local CPU; threshold 0.6 and 800 ms silence |
 
 To select another environment:
 
@@ -302,7 +323,15 @@ APP_ENV=staging uv run uvicorn \
   --host 127.0.0.1 --port 8001
 ```
 
-The `staging` and `production` files are present, but deployment readiness for those environments has not been established. When services run in different containers or machines, replace loopback addresses with reachable service endpoints.
+The `staging` and `production` files are currently empty placeholders. Populate
+the selected file before using it; the loader rejects an empty YAML. Start from
+`development.yaml`, then configure reachable provider endpoints and VAD/input
+settings for that environment. Deployment readiness has not been established.
+
+TTS voice selection is `tts.options.voice` in the selected environment YAML.
+The service determines which voice identifiers are supported. Configuration is
+loaded at Gateway startup: restart the process and open a new conversation after
+changing YAML; refreshing the browser alone does not reload server configuration.
 
 ## WebSocket protocol
 
@@ -344,7 +373,8 @@ STT events may arrive while input audio is still streaming. In server detection
 
 ### Interruption and session closure
 
-- `interrupt` targets a `turn_id`. For a matching active turn, the Gateway cancels and awaits execution, aborts STT, releases turn state, and then acknowledges with `interrupted`.
+- In server detection mode, `SpeechStarted` makes `VoiceSession` interrupt an active turn before opening a new one. The Gateway maps application events to `interrupted` and `turn.started`; the client stops obsolete playback.
+- Explicit `interrupt` targets a `turn_id`. For a matching active turn, `VoiceSession` cancels and awaits execution, aborts STT, and releases turn state. The Gateway then delivers the acknowledgement.
 - `session.end` requests cleanup and session closure; successful completion is acknowledged with `session.ended`.
 - Errors are returned as structured `error` events (for example, `STT_START_FAILED` or `TURN_CANCEL_FAILED`).
 
@@ -352,7 +382,7 @@ STT events may arrive while input audio is still streaming. In server detection
 
 The system separates **semantic state** from **asynchronous execution**. `TurnManager` controls listening, processing, speaking, and completion; `VoicePipeline` owns the registered turn task. VoiceSession coordinates provider shutdown; the Gateway owns WebSocket delivery and shields cleanup from transport cancellation.
 
-Cancellation is cooperative: `task.cancel()` requests termination, and the Gateway awaits task completion before confirming a successful interruption. Startup rollback attempts to cancel any created task and abort a partially initialized STT provider. Failed cleanup is reported rather than represented as a successful retry.
+Cancellation is cooperative: `task.cancel()` requests termination, and `VoiceSession` awaits task completion before reporting a successful interruption. Startup rollback attempts to cancel any created task and abort a partially initialized STT provider. Failed cleanup is reported rather than represented as a successful retry.
 
 ## Testing
 
@@ -369,6 +399,12 @@ uv run pytest -q tests/test_gateway.py
 uv run ruff check src tests --select F401,F811,F821
 ```
 
+Run browser controller, worklet, and playback tests:
+
+```bash
+node --test tests/js/*.test.mjs
+```
+
 ### End-to-end smoke test
 
 With the Gateway, NVIDIA ASR, and VoxCPM services available:
@@ -377,9 +413,36 @@ With the Gateway, NVIDIA ASR, and VoxCPM services available:
 uv run python tools/voice_pipeline_smoke_test.py
 ```
 
-The script reads `audios/chafik_normalize.wav` (16 kHz, mono, 16-bit PCM), sends audio over WebSocket, and checks for a final transcript, an Agent response, synthesized audio, and `turn.ended`. It writes raw output to `/tmp/voice_pipeline_response.pcm`.
+This existing script uses manual turn control, not continuous server VAD.
+Its `AUDIO_FILE` currently points to the absolute path
+`/home/ubuntu/voice-assistant/audios/chafik_normalize.wav`; provide a real
+16 kHz, mono, 16-bit PCM WAV and update that constant before running it elsewhere.
+`GATEWAY_URL` must match the running port. It checks for a final transcript, an
+Agent response, synthesized audio, and `turn.ended`, and writes raw output to
+`/tmp/voice_pipeline_response.pcm`. It does not assess recognition accuracy.
 
-**Last verified checkpoint:** 142 automated tests passing; the full NVIDIA STT → Mock Agent → VoxCPM TTS smoke test completed successfully. These results describe the development environment, not a production SLA.
+### Continuous conversation checks and tuning
+
+With the browser console open, test initial silence, background noise, a short
+phrase, a longer phrase with natural pauses, interruption during playback, and
+session closure/reconnection. Compare `[STT FINAL]` with the words actually spoken;
+hearing synthesized speech alone does not confirm recognition accuracy.
+
+During development, the phrase "fala um oi pra Lara" was transcribed with a
+missing ending at 300 ms silence and recognized correctly after increasing it to
+800 ms. Threshold was then increased from 0.5 to 0.6 for a background-noise trial.
+These are environment-specific adjustments, not universal recommended values.
+The current application interrupts on `SpeechStarted`, without an additional
+minimum sustained-speech confirmation. False positives can therefore interrupt
+playback; background speech and acoustic echo need separate evaluation.
+
+Functional conversation checks do not establish a WER benchmark or resource
+budget. Systematic WER, end-to-end latency distributions, and CPU/memory under
+concurrent sessions have not been measured. A repeatable quality evaluation needs
+recorded audio plus reference transcripts, with consistent normalization and
+identical input for each configuration. Measure response latency to audible
+playback separately from server synthesis completion. Results from the Mock
+Agent do not characterize latency with a production LLM.
 
 ## Project structure
 
@@ -448,7 +511,9 @@ Each adapter owns its model state, PCM buffer, and input timeline. It converts
 PCM S16LE to normalized float samples, buffers 512-sample windows, and maps
 Silero boundaries to the application events. Defaults are threshold 0.5,
 300 ms of silence, 30 ms speech padding, and a 10-second worker wait limit.
-These values are initial settings, to be tuned with real conversational audio.
+These constructor defaults apply when options are omitted. The development YAML
+overrides threshold to 0.6 and silence to 800 ms; tune them with representative
+audio rather than changing constructor defaults.
 
 `process()` may return several events for a large chunk. `flush()` pads a final
 partial window internally while keeping event positions within real audio.
@@ -482,8 +547,8 @@ structure as STT, Agent, and TTS. `config/development.yaml` contains:
 vad:
   provider: silero
   options:
-    threshold: 0.5
+    threshold: 0.6
-    min_silence_duration_ms: 300
+    min_silence_duration_ms: 800
     speech_pad_ms: 30
     worker_timeout_seconds: 10.0
 ```
@@ -499,6 +564,20 @@ new adapter instances after configuration is reloaded; there is no live reload.
 The main web client uses the VAD factory through continuous voice sessions.
 Its detection settings come from the active environment YAML.
 
+| Option | Development value | Effect |
+| --- | --- | --- |
+| `vad.options.threshold` | `0.6` | Speech probability activation threshold. Raising it can reduce false starts but miss quiet speech. |
+| `vad.options.min_silence_duration_ms` | `800` | Silence required to end detected speech. Longer waits preserve pauses but delay finalization. |
+| `vad.options.speech_pad_ms` | `30` | Padding around detected boundaries; does not confirm sustained speech. |
+| `vad.options.worker_timeout_seconds` | `10.0` | Maximum wait for a model worker operation; not an endpointing delay or forced thread termination. |
+| `voice_input.pre_roll_ms` | `200` | Audio retained before the detected start and forwarded to STT, bounded by history and the previous utterance. |
+| `voice_input.history_ms` | `2000` | Retained input history for delayed boundary events; must cover pre-roll. |
+| `voice_input.max_input_chunk_bytes` | `65536` | Maximum continuous PCM input chunk, also capped by the Gateway at 64 KiB. |
+
+Tune one parameter at a time using the same audio. End-of-speech silence does
+not filter start-of-speech noise triggers. No additional interruption-confirmation
+parameter is implemented; that would require an application policy change.
+
 
 ### Continuous audio and backend turn control