feat: integra VAD ao controle de turnos da conversa
This commit is contained in:
58
README.md
58
README.md
@@ -388,7 +388,7 @@ The script reads `audios/chafik_normalize.wav` (16 kHz, mono, 16-bit PCM), sends
|
||||
config/ Environment YAML and provider catalog
|
||||
src/voice_assistant/
|
||||
audio_gateway/app/ WebSocket endpoint and protocol
|
||||
application/ VoiceSession and transport-independent events
|
||||
application/ VoiceSession, continuous PCM history, and application events
|
||||
domain/ Session state and domain rules
|
||||
vad/ VAD port, speech boundaries, and local Silero adapter
|
||||
audio/ Canonical audio models
|
||||
@@ -418,7 +418,7 @@ tools/ Integration and smoke scripts
|
||||
|
||||
|
||||
|
||||
## Backend VAD contract (integration pending)
|
||||
## Backend VAD contract
|
||||
|
||||
`vad/adapter.py` defines one stateful detector per continuous input stream.
|
||||
Its input is the canonical PCM S16LE, mono, 16 kHz audio format. Arbitrary
|
||||
@@ -432,9 +432,9 @@ boundary. Model padding never extends the input timeline.
|
||||
`reset()` discards detection state and restarts the timeline, and `aclose()`
|
||||
releases resources without emitting new speech events. Calls are serialized.
|
||||
These contracts do not start turns, select providers, or carry transport IDs.
|
||||
The local Silero implementation is available independently of the running
|
||||
conversation. Application integration and browser VAD replacement are subsequent
|
||||
tasks; the active conversation flow still uses its existing browser VAD.
|
||||
The Silero adapter can control turns through the continuous VoiceSession API.
|
||||
Browser/WebSocket migration is the next task; the existing web client continues
|
||||
to use its current turn commands until that migration.
|
||||
|
||||
|
||||
### Local Silero VAD adapter
|
||||
@@ -497,5 +497,49 @@ remains compatible with existing configurations; explicitly creating VAD in
|
||||
such an environment raises a configuration error. Changing YAML values affects
|
||||
new adapter instances after configuration is reloaded; there is no live reload.
|
||||
|
||||
This factory makes the configuration usable independently. Connecting it to the
|
||||
conversation lifecycle remains the next task.
|
||||
The VAD factory can be supplied to continuous voice sessions. The existing web
|
||||
client remains on manual turn commands until the browser migration.
|
||||
|
||||
|
||||
### Continuous audio and backend turn control
|
||||
|
||||
A `VoiceSession` constructed with a VAD receives continuous canonical PCM through
|
||||
`await session.receive_audio(pcm)`. The caller supplies audio without a turn ID;
|
||||
the channel owns transport sequencing. `SpeechStarted` opens an STT turn and
|
||||
`SpeechEnded` schedules normal STT finalization in the background. New speech
|
||||
interrupts an active response/finalization before opening a fresh turn. Empty
|
||||
STT results release the turn through an interruption event rather than leaving
|
||||
an active turn behind. The VAD is not reset between conversation turns.
|
||||
|
||||
The application's sample-indexed audio history preserves the beginning of speech
|
||||
when VAD confirmation arrives later. PCM already sent to a turn is not replayed
|
||||
within that turn; pre-roll never includes the previous utterance. History and
|
||||
input size are bounded by environment configuration:
|
||||
|
||||
```yaml
|
||||
voice_input:
|
||||
pre_roll_ms: 200
|
||||
history_ms: 2000
|
||||
max_input_chunk_bytes: 65536
|
||||
```
|
||||
|
||||
History is trimmed after processing each input chunk; transient storage is at most
|
||||
retained history plus one accepted chunk. A speech start older than retained
|
||||
history is reported as an input error instead of silently dropping speech.
|
||||
Invalid PCM is rejected before detection or timeline advancement. A processing
|
||||
failure requires a new session; cleanup still releases VAD and provider resources.
|
||||
|
||||
Use `finish_audio_input()` for normal end of a continuous stream. Session end or
|
||||
disconnect aborts through `end()`/`aclose()`, without forcing a final speech event.
|
||||
The caller shields cleanup from transport cancellation, as for manual sessions.
|
||||
|
||||
Backend-controlled sessions emit `TurnOpened` and `TurnInterrupted`; the WebSocket
|
||||
mapper translates them to the existing `turn.started` and `interrupted` events.
|
||||
Adapters should discard outdated queued playback when a new turn opens: backend
|
||||
turn completion describes produced output, not physical speaker completion.
|
||||
|
||||
Manual `start_turn`/`send_audio`/`stop_turn` commands remain available on sessions
|
||||
without VAD. Mixing those commands with a VAD-controlled session is rejected.
|
||||
`create_voice_session(..., server_vad=True)` composes the continuous backend API;
|
||||
the current `/ws/audio` receive loop still uses the manual mode. The next task
|
||||
connects continuous input to that route and simplifies the browser accordingly.
|
||||
|
||||
Reference in New Issue
Block a user