Voice Activity Detection & Turn-Taking — Silero, Cobra, and the Flush Trick

A voice agent makes three distinct decisions on every 20 ms chunk:

1. Is this frame speech? That's VAD: binary, per-frame.
2. Has the user started a new utterance? Onset detection.
3. Has the user finished? End-pointing, also called turn-end.

A naive energy threshold fails the moment real noise shows up — traffic, keyboards, crowd babble. The 2026 answer stacks three things instead: Silero VAD (open, deep-learned), a turn-detection model for semantic endpointing, and a VAD-calibrated silence hangover.

The three-tier VAD cascade

Tier 1: energy gate. Cheapest. Threshold RMS at -40 dBFS. Filters obvious silence but fires on any noise above the threshold.

Tier 2: Silero VAD (2020-2026, MIT). 1M parameters. Trained on 6000+ languages. Runs in ~1 ms per 30 ms chunk on a single CPU thread. 87.7% TPR at 5% FPR. The open-source default.

Tier 3: semantic turn detector. LiveKit's turn-detection model (2024-2026) or your own small classifier. Distinguishes "pause mid-sentence" from "done talking." Uses linguistic context (intonation + recent words), not just silence.

Key parameters and their defaults

• Threshold. Silero outputs a probability; classify speech at > 0.5 (default) or > 0.3 (sensitive). Lower threshold = fewer first-word clips, more false positives.
• Minimum speech duration. Reject speech shorter than 250 ms (usually coughs or chair noise).
• Silence hangover (end-pointing). After VAD returns to 0, wait 500-800 ms before declaring end-of-turn. Too short → interrupt user. Too long → feels sluggish.
• Pre-roll buffer. Keep 300-500 ms of audio before VAD fires. Prevents "hey" being clipped.

The flush trick (Kyutai 2025)

Streaming STT models have a look-ahead delay (500 ms for Kyutai STT-1B, 2.5 s for STT-2.6B). Normally you'd sit through that whole delay after the user stops talking before a transcript shows up. The flush trick fixes that: the moment VAD fires end-of-speech, send a flush signal to the STT and force immediate output. STT processes at ~4× realtime, so the 500 ms buffer finishes in ~125 ms.

End-to-end: 125 ms VAD + flush STT = conversational latency.

2026 VAD comparison

Silero is the right default. Cobra is the compliance / accuracy upgrade. Energy-only VAD has no place in 2026 production.

Step 1: the energy gate

def energy_vad(chunk, threshold_dbfs=-40.0):
    rms = (sum(x * x for x in chunk) / len(chunk)) ** 0.5
    dbfs = 20.0 * math.log10(max(rms, 1e-10))
    return dbfs > threshold_dbfs

Step 2: Silero VAD in Python

from silero_vad import load_silero_vad, get_speech_timestamps
 
vad = load_silero_vad()
audio = torch.tensor(waveform_16k, dtype=torch.float32)
segments = get_speech_timestamps(
    audio, vad, sampling_rate=16000,
    threshold=0.5,
    min_speech_duration_ms=250,
    min_silence_duration_ms=500,
    speech_pad_ms=300,
)
for s in segments:
    print(f"{s['start']/16000:.2f}s - {s['end']/16000:.2f}s")

Step 3: turn-end state machine

class TurnDetector:
    def __init__(self, silence_hangover_ms=500, min_speech_ms=250):
        self.state = "idle"
        self.speech_ms = 0
        self.silence_ms = 0
        self.silence_hangover_ms = silence_hangover_ms
        self.min_speech_ms = min_speech_ms
 
    def update(self, is_speech, chunk_ms=20):
        if is_speech:
            self.speech_ms += chunk_ms
            self.silence_ms = 0
            if self.state == "idle" and self.speech_ms >= self.min_speech_ms:
                self.state = "speaking"
                return "START"
        else:
            self.silence_ms += chunk_ms
            if self.state == "speaking" and self.silence_ms >= self.silence_hangover_ms:
                self.state = "idle"
                self.speech_ms = 0
                return "END"
        return None

Step 4: the flush trick skeleton

def flush_on_end(stt_client, audio_buffer):
    stt_client.send_audio(audio_buffer)
    stt_client.send_flush()
    return stt_client.recv_transcript(timeout_ms=150)

STT providers like Kyutai, Deepgram, and AssemblyAI support flush; Whisper streaming doesn't, because it's block-based and always waits for full chunks.

Rule of thumb: don't ship energy-only VAD unless you genuinely have no other option.

• Fixed threshold. Works in a quiet room, falls apart the moment real noise shows up. Calibrate it on-device, or just switch to Silero.
• Too-short silence hangover. The agent interrupts mid-sentence. 500-800 ms is the sweet spot for conversational speech.
• Too-long hangover. It feels sluggish. A/B test against real target users to find the line.
• No pre-roll buffer. You lose the first 200-300 ms of user audio. Always keep a rolling pre-roll.
• Ignoring semantic endpointing. "Hmm, let me think..." contains long pauses, and users hate being cut off mid-thought. Use LiveKit's turn-detector or something similar.

Save it as outputs/skill-vad-tuner.md. Pick a VAD model, threshold, hangover, pre-roll, and turn-detection strategy for your workload.

1. Easy. Run code/main.py: it simulates a speech-silence-speech-coughs sequence and tests three VAD tiers.
2. Medium. Install silero-vad, process a 5-minute recording, and tune the threshold to minimize both first-word clips and false triggers. Report precision and recall.
3. Hard. Build a mini turn-detector: Silero VAD plus a 3-layer MLP over the last 10 words' embeddings (use sentence-transformers). Train it on a hand-labeled turn-end dataset. Beat Silero alone by 10% F1.

• Silero VAD: the reference open VAD.
• Picovoice Cobra VAD: the commercial accuracy leader.
• Kyutai Unmute, the flush trick: the sub-200 ms engineering trick.
• LiveKit turn detection: semantic endpointing in production.
• WebRTC VAD: the legacy baseline.
• pyannote segmentation: diarization-grade segmentation.