refactor: define contrato de VAD e eventos de fala

This commit is contained in:
alex alves
2026-11-03 17:01:03 -03:00
parent 76d15b8e52
commit 2d1acdac31
4 changed files with 118 additions and 0 deletions

View File

@@ -390,6 +390,7 @@ src/voice_assistant/
audio_gateway/app/ WebSocket endpoint and protocol
application/ VoiceSession and transport-independent events
domain/ Session state and domain rules
vad/ VAD port and speech boundaries; runtime integration pending
audio/ Canonical audio models
stt/ STT contract, mock and NVIDIA adapter
agent/ Agent contract and mock adapter
@@ -415,3 +416,21 @@ tools/ Integration and smoke scripts
**Status:** V1 development baseline · Python 3.12+ · WebSocket protocol 1.0
## Backend VAD contract (integration pending)
`vad/adapter.py` defines one stateful detector per continuous input stream.
Its input is the canonical PCM S16LE, mono, 16 kHz audio format. Arbitrary
complete PCM frames are buffered into model-specific windows by the adapter.
`vad/models.py` defines `SpeechStarted` and `SpeechEnded`; their sample offsets
refer to input audio since initialization or reset, with inclusive starts
and exclusive ends. Detection confirmation may arrive later than its audio
boundary. Model padding never extends the input timeline.
`process()` returns ordered boundaries, `flush()` handles normal end of input,
`reset()` discards detection state and restarts the timeline, and `aclose()`
releases resources without emitting new speech events. Calls are serialized.
These contracts do not start turns, select providers, or carry transport IDs.
The Silero implementation, application integration, and browser VAD replacement
are subsequent tasks. This step leaves the active conversation flow unchanged.

View File

@@ -0,0 +1 @@
"""Voice activity detection contracts independent of model and channel."""

View File

@@ -0,0 +1,71 @@
"""Port for stateful, continuous voice activity detection."""
from abc import ABC, abstractmethod
from voice_assistant.audio.models import AudioFormat, CANONICAL_AUDIO_FORMAT
from voice_assistant.vad.models import VADEvent
class VADAdapter(ABC):
"""One detector per continuous input stream, used sequentially.
Audio enters as PCM S16LE, mono, 16 kHz. Adapters buffer arbitrary
complete PCM frames into the windows required by their model, convert
samples internally, and return only speech-boundary events.
Model state must not be shared between streams.
Each stream starts outside speech, at sample zero. Offsets refer to
actual input audio; synthetic padding must not extend the timeline.
Events are ordered by audio position and alternate start/end. A call
may return no events or multiple boundaries when it consumes a large
chunk. Detection latency does not change a boundary's sample offset.
The caller serializes operations. Implementations must avoid blocking
the event loop and must retain ownership of any worker still running
after cancellation, so it cannot race with reset or resource cleanup.
"""
@property
def input_format(self) -> AudioFormat:
return CANONICAL_AUDIO_FORMAT
@abstractmethod
async def process(self, audio: bytes) -> tuple[VADEvent, ...]:
"""Consume PCM once and return new speech boundaries.
Validate PCM before changing state. Empty or misaligned chunks
raise ValueError. Chunk sizes are independent of model windows.
Calls after flush require reset; calls after aclose fail.
Model errors and cancellation propagate to the caller.
"""
raise NotImplementedError
@abstractmethod
async def flush(self) -> tuple[VADEvent, ...]:
"""Finish input normally, processing any remaining partial window.
If speech remains active, emit SpeechEnded with reason
end_of_input at the last real sample's exclusive boundary.
A repeated flush returns no events. New audio requires reset.
This operation does not release the detector's resources.
"""
raise NotImplementedError
@abstractmethod
async def reset(self) -> None:
"""Discard buffered audio and detection state; restart at sample zero.
Emit no boundaries. Use for an explicit stream discontinuity,
rather than after every conversation turn. Reset after aclose fails.
"""
raise NotImplementedError
@abstractmethod
async def aclose(self) -> None:
"""Release resources and discard pending audio without emitting events.
Safe to repeat, including after partial initialization. Use this
operation for abort/disconnect; flush may produce a final speech
event and is reserved for normal end of input.
"""
raise NotImplementedError

View File

@@ -0,0 +1,27 @@
"""Speech boundaries on the input audio timeline, independent of turns."""
from dataclasses import dataclass
from typing import Literal
@dataclass(frozen=True, slots=True)
class SpeechStarted:
"""Inclusive speech start, in samples since initialization or reset."""
sample_offset: int
@dataclass(frozen=True, slots=True)
class SpeechEnded:
"""Exclusive speech end on the same timeline as SpeechStarted.
The offset describes the audio boundary, not the later instant at which
enough silence has arrived to confirm it. Synthetic end-of-input padding
must not extend the offset beyond actual input audio.
"""
sample_offset: int
reason: Literal["silence", "end_of_input"]
type VADEvent = SpeechStarted | SpeechEnded