TAI BUI
Back to curriculum
BuildPython

Voice Anti-Spoofing & Audio Watermarking — ASVspoof 5, AudioSeal, WaveVerify

Voice cloning shipped years before anyone built defenses for it. A production voice system in 2026 needs two things: a detector (AASIST, RawNet2) that tells real speech from fake, and a watermark (AudioSeal) that survives compression and editing. Ship both, or don't ship voice cloning at all.

TIME: ~75 minutesPREREQUISITES: Phase 6 · 06 (Speaker Recognition), Phase 6 · 08 (Voice Cloning)

Learning Objectives

  • Name the three defenses a production voice system needs in 2026 (anti-spoofing detection, audio watermarking, and authenticated provenance) and explain which threat each one covers.
  • Read an ASVspoof 5 EER number and explain what changed in the benchmark's data, speaker count, and attack coverage versus earlier editions.
  • Embed and detect an AudioSeal watermark in code, and explain why a watermark with no detector running, or a C2PA manifest with no watermark, each fail on their own.
  • Match a detection model, watermark, and provenance approach to a specific voice-system use case: shipping TTS, biometric unlock, fraud detection, or provenance signing.

The Problem

Voice cloning shipped years ahead of any way to catch it, so a production voice system now needs three separate defenses layered together.

  1. Anti-spoofing / deepfake detection. Given an audio clip, is it synthetic or real? The ASVspoof benchmarks (2019, then 2021, now ASVspoof 5) are still the reference point.
  2. Audio watermarking. Embed a signal in generated audio that's inaudible to a listener but recoverable by a detector later. AudioSeal (Meta) and WavMark are the open options.
  3. Authenticated provenance. Cryptographically sign the audio file and its metadata, in the C2PA / Content Authenticity Initiative format.

Detection is for the adversary who won't cooperate. Watermarking is for the honest case, marking AI-generated audio as AI-generated for compliance. Both are required in 2026.

Warm-up0/3
1. In a speaker verification system, what is a false accept?Warm-up
A voice-unlock system with Alice enrolled. Attempt 1 is Alice speaking, attempt 2 is Bob, who is not enrolled. Each attempt goes into the verification system and ends in a decision that is left as a question mark.
Schematic of two attempts; decisions left open.
2. An audio file is described as sampled at 16 kHz. What does that mean?Warm-up
A wavy audio line with orange dots marking stored measurements and a bracket underneath marking one second of audio. A dashed box asks how many samples fall inside that second.
Schematic; the dots are drawn much sparser than a real file.
3. What does lossy compression such as MP3 do to an audio file?Warm-up
A pipeline: an original waveform goes into an MP3 encode step and an MP3 decode step, and comes out as a decoded waveform shown as a dashed question box to compare with the original.
Schematic of the round trip; the result is left open.
Check: The Problem0/3
1. What three related defenses does the lesson identify for voice-cloning safety?Define
A flow from a voice sample into a cloning model, to a generated clip, to people hearing it. Below it sit three dashed boxes labelled defense 1, defense 2 and defense 3, each with a question mark.
Schematic; the three defenses are left blank.
2. Per the lesson, what's the difference in what detection versus watermarking handle?Explain
On the left, a provider that follows the rules and an attacker who generates audio elsewhere. On the right, two dashed boxes: what detection handles and what watermarking handles, both left open.
Schematic; the pairing is left open.
3. Why is watermarking aimed at the honest case rather than the adversary?Explain
Audio from Generator X, a service of your own, and Generator Y, run by an attacker, both go to a watermark detector. The results for X audio and Y audio are dashed question boxes.
Schematic; detector results are left open.

The Concept

Anti-spoofing vs watermarking vs provenance — three defense layers

ASVspoof 5: the 2024-2025 benchmark

Here's what changed from prior editions:

  • Crowdsourced data, not studio-clean recordings, so conditions match what a detector actually sees in production.
  • ~2000 speakers, up from ~100 in earlier editions.
  • 32 attack algorithms, spanning TTS, voice conversion, and adversarial perturbation.
  • Two tracks: Countermeasure (CM) for standalone detection, and Spoofing-robust ASV (SASV) for biometric systems.

State of the art on ASVspoof 5 sits around 7.23% EER (equal error rate: the point where a detector's false-accept rate and false-reject rate cross, so lower is better). On the older ASVspoof 2019 LA set it's 0.42% EER. In the wild, on real deployment traffic, expect 5-10% EER instead.

AASIST and RawNet2: the detection model families

AASIST (2021, updated through 2026) runs graph attention over spectral features and is the current SOTA on the ASVspoof 5 countermeasure task.

RawNet2 pairs a convolutional front-end over the raw waveform with a TDNN backbone. It's the simpler baseline, and still competitive once you fine-tune it.

NeXt-TDNN plus SSL features, a 2025 variant, combines an ECAPA-style architecture with WavLM features and focal loss. That combination is what reaches the 0.42% EER figure on ASVspoof 2019 LA quoted above.

AudioSeal: the 2024 watermark default

Meta shipped AudioSeal in January 2024, with v0.2 following in December 2024. Five choices define the design:

  • Localized. It detects the watermark per frame, at 16 kHz sample resolution (1/16000 s).
  • Generator and detector jointly trained. The generator learns to embed an inaudible signal; the detector learns to find it by training through augmentations.
  • Robust. It survives MP3/AAC compression, EQ, speed-shift of ±10%, and a noise mix at +10 dB SNR.
  • Fast. The detector runs at 485× realtime, about 1000× faster than WavMark.
  • High capacity. A 16-bit payload, enough to encode a model ID, generation timestamp, and user ID, embeds in every utterance.

WavMark

WavMark was the open baseline before AudioSeal: an invertible neural network carrying 32 bits/sec. It has real problems, though:

  • Synchronization relies on brute force, and that's slow.
  • Gaussian noise or MP3 compression can strip it out.
  • It isn't real-time friendly.

WaveVerify (July 2025)

WaveVerify targets exactly what AudioSeal struggles with: temporal manipulation, meaning reversal and speed changes. It pairs a FiLM-based generator with a Mixture-of-Experts detector, and is competitive with AudioSeal on standard attacks while also holding up under temporal edits.

The gap adversaries exploit

AudioMarkBench found the gap all of these share: "under pitch shift, all watermarks show Bit Recovery Accuracy below 0.6, indicating near-complete removal." Pitch-shift is the universal attack. No 2026 watermark fully resists aggressive pitch modification, which is exactly why you pair watermarking with detection like AASIST rather than relying on either alone.

C2PA / Content Authenticity Initiative

C2PA isn't an ML technique. It's a manifest format: audio files carry cryptographically signed metadata about the creation tool, author, and date. Meta's own Audiobox and Seamless demos ship AudioSeal watermarks, not C2PA manifests, which is a sign of how far behind images audio-side C2PA adoption still is. It's good for provenance, but it does nothing once a bad actor re-encodes the file and strips the metadata.

Check: The Concept0/20
1. What is AASIST's detection approach, per the lesson?Define
An audio clip goes into a box named AASIST whose inner approach is a question mark, and out comes a score for real versus synthetic.
Schematic; the inside of the box is left open.
2. How does the lesson describe RawNet2 and the 2025 NeXt-TDNN variant?Define
Two rows, RawNet2 and NeXt-TDNN (2025). Each is a chain of an input box, two dashed question boxes, and a real-versus-fake output box.
Schematic; the internals are left open.
3. What is ASVspoof?Define
Three boxes in a row named ASVspoof 2019, ASVspoof 2021 and ASVspoof 5, with a dashed question box below asking what kind of thing ASVspoof is.
Names as they appear in the lesson.
4. What is the equal error rate (EER) of a detector?Define
A score axis from 0 to 1 with five blue dots for real clips at 0.35, 0.6, 0.7, 0.8 and 0.9, and five red dots for fake clips at 0.1, 0.2, 0.3, 0.55 and 0.65. A dashed box asks for the EER of this detector.
Illustrative scores, typed by hand; not from a real detector.
5. What is a countermeasure (CM) in anti-spoofing?Define
An audio clip of unknown origin goes into a box labelled CM for countermeasure, and the output is a dashed question box.
Schematic; the output is left open.
6. What does the SASV track of ASVspoof 5 evaluate?Define
Two dashed boxes side by side, one labelled CM track and one labelled SASV track, each with an open question about what it evaluates.
Track names from the lesson; their tasks are left open.
7. What is AudioSeal?Define
An import line for AudioSeal, then generated.wav going into a box named AudioSeal (January 2024), with a dashed question box for what comes out.
The import line and date are from the lesson; the output is left open.
8. What does Bit Recovery Accuracy measure?Define
An eight-bit payload 1 0 1 1 0 1 0 0 is embedded, an attack is applied, and a decoded row 1 0 1 0 0 1 0 0 comes out. A dashed box asks for the Bit Recovery Accuracy.
Illustrative 8-bit payload, typed by hand; not from a real watermark.
9. What is C2PA when used for audio?Define
One file drawn as a container holding a box for the audio samples and a dashed box labelled C2PA whose contents are a question mark.
Schematic layout; contents are left open.
10. What does the lesson give as AudioSeal's detector speed relative to WavMark?Define
One audio clip goes to a WavMark detector with time t1 and an AudioSeal detector with time t2. A dashed box asks for the ratio t1 over t2.
Schematic; no timings are shown.
11. According to AudioMarkBench as cited in the lesson, what is the 'universal attack' that defeats nearly all watermarks?Define
A Bit Recovery Accuracy axis from 0.5 to 1.0 with the region below 0.6 shaded and labelled as where all watermarks fall under one attack. A dashed box asks which attack it is.
Axis and 0.6 line from the lesson's AudioMarkBench quote.
12. What EER numbers does the lesson report for ASVspoof 5, the older ASVspoof 2019 LA, and real-world in-the-wild clips?Define
Three columns: ASVspoof 5 (newest benchmark), ASVspoof 2019 LA (older benchmark) and in-the-wild clips (real deployment traffic). Each has a dashed box reading EER equals a question mark.
Values left open.
13. What changed in ASVspoof 5 compared to prior editions, per the lesson?Explain
A comparison table with rows for recording conditions, speakers, attack algorithms and tracks, and columns for earlier editions and ASVspoof 5, every value a question mark.
Row labels only; values are left open.
14. Why is AudioSeal's generator trained jointly with its detector, through augmentations?Explain
A training loop: the generator makes marked audio, an augmentation step edits it, the detector looks for the mark, and a dashed feedback line returns to both. A dashed box asks why it is trained this way.
Schematic of the lesson's description.
15. A detector scores 0.42% EER on ASVspoof 2019 LA but the lesson expects 5-10% EER on real traffic. What explains the gap?Explain
One detector scored on ASVspoof 2019 LA, giving EER 0.42 percent, and on real traffic, giving EER 5 to 10 percent.
Numbers from the question stem.
16. Why does the lesson pair watermarking with a detector like AASIST instead of relying on the watermark alone?Explain
An audio clip goes to two checks side by side, a watermark detector and AASIST, and both feed a dashed box asking why both are used.
Schematic; the reason is left open.
17. Why is WavMark a poor fit for real-time use compared with AudioSeal?Explain
A row of eight audio frames f1 to f8 arriving in a stream, and two dashed boxes for WavMark and AudioSeal, each asking how it finds the mark.
Schematic; both methods are left open.
18. A user reverses and speeds up a watermarked clip before reposting it, and the watermark needs to survive that edit. Which 2025 watermark family was built for exactly this, and how does it work?Apply
A waveform that starts loud and fades, shown as a watermarked clip. Below it, the same shape mirrored and shortened to show a reversed and sped-up edit. On the right, a dashed box asks which watermark family fits.
Waveforms are hand-drawn schematics.
19. An attacker pitch-shifts a generated clip aggressively. AudioSeal's detector returns a low presence score, yet AASIST flags the clip as synthetic. How should this be read?Analyze
A generated clip that was pitch-shifted goes to the AudioSeal detector, which returns a low presence score, and to AASIST, which flags the clip as synthetic. A dashed box asks how to read the two results.
Results are those given in the question stem.
20. A clip loses its C2PA manifest after being re-encoded, yet the AudioSeal detector still decodes its payload after MP3 compression. What explains the difference?Analyze
A table of a clip before and after re-encoding. The C2PA manifest is present and then lost. The AudioSeal payload is present and still decodes after MP3 compression. A dashed box asks what explains the difference.
Statuses are those given in the question stem.

Build It

Step 1: a simple spectral-feature detector (toy)

Synthetic speech tends to have unusually flat high-frequency energy compared to a real recording, and that's the intuition a cheap spectral-rolloff check can lean on:

python
def spectral_rolloff(spec, percentile=0.85):
    cum = 0
    total = sum(spec)
    if total == 0:
        return 0
    threshold = total * percentile
    for k, v in enumerate(spec):
        cum += v
        if cum >= threshold:
            return k
    return len(spec) - 1
 
def is_suspicious(audio):
    spec = magnitude_spectrum(audio)
    rolloff = spectral_rolloff(spec)
    return rolloff / len(spec) > 0.92

Production systems use AASIST instead, which works from a much richer set of spectro-temporal features rather than one rolloff heuristic. The basic intuition still holds, though: synthesis leaves a spectral fingerprint, and a detector's job is to find it.

Step 2: AudioSeal embed + detect

python
from audioseal import AudioSeal
import torch
 
generator = AudioSeal.load_generator("audioseal_wm_16bits")
detector = AudioSeal.load_detector("audioseal_detector_16bits")
 
audio = load_wav("generated.wav", sr=16000)[None, None, :]
payload = torch.tensor([[1, 0, 1, 1, 0, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1, 0]])
watermark = generator.get_watermark(audio, sample_rate=16000, message=payload)
watermarked = audio + watermark
 
result, decoded_payload = detector.detect_watermark(watermarked, sample_rate=16000)
# result: float in [0, 1] — probability of watermark presence
# decoded_payload: 16 bits; match against embedded payload

Step 3: measuring detector quality with EER

python
def eer(real_scores, fake_scores):
    thresholds = sorted(set(real_scores + fake_scores))
    best = (float("inf"), 0.0)
    for t in thresholds:
        far = sum(1 for s in fake_scores if s >= t) / len(fake_scores)
        frr = sum(1 for s in real_scores if s < t) / len(real_scores)
        if abs(far - frr) < best[0]:
            best = (abs(far - frr), (far + frr) / 2)
    return best[1]

Step 4: the production integration

python
def safe_tts(text, voice, clone_reference=None):
    if clone_reference is not None:
        verify_consent(user_id, clone_reference)
    audio = tts_model.synthesize(text, voice)
    audio_with_wm = audioseal_embed(audio, payload=build_payload(user_id, model_id))
    manifest = c2pa_sign(audio_with_wm, user_id, timestamp=now())
    return audio_with_wm, manifest

Every generation that leaves this pipeline ships with three things: (1) a watermark, (2) a signed manifest, (3) a retention-policy-compliant audit log.

Check: Build It0/6
1. In the lesson's `safe_tts` production integration example, what three things does every generation ship with?Apply
The safe_tts signature, then inputs text and voice going into safe_tts, and out comes a dashed box reading ships with 3 things, numbered 1, 2 and 3, each a question mark.
Signature from the lesson; the three items are left open.
2. Using the `eer` function with real_scores = [0.1, 0.2, 0.5, 0.6] and fake_scores = [0.4, 0.7, 0.8, 0.9], what are far and frr at threshold t = 0.5?Apply
A score axis from 0 to 1 with a dashed threshold line at t equals 0.5. Four blue dots for real scores at 0.1, 0.2, 0.5 and 0.6, and four red dots for fake scores at 0.4, 0.7, 0.8 and 0.9, with far and frr left as question marks.
Plotted from the stem's lists.
3. In `spectral_rolloff`, spec = [1, 1, 1, 1, 1, 1, 1, 1, 1, 1] with the default percentile. What index is returned, and does `is_suspicious` flag it?Apply
Ten spectrum bins at indices 0 to 9, each with value 1. Below, the spectral_rolloff signature with percentile 0.85 and the is_suspicious return line comparing rolloff over len(spec) to 0.92. Two dashed boxes ask for the returned index and whether it is flagged.
Function names and constants are from the lesson; the body is elided.
4. The Build It payload is [1,0,1,1,0,1,0,0,1,1,0,1,0,1,1,0]. After an attack the detector decodes it with exactly two bits flipped. What is the Bit Recovery Accuracy?Apply
The 16-bit payload 1 0 1 1 0 1 0 0 1 1 0 1 0 1 1 0, an attack box saying exactly two bits are flipped at decode, a decoded row of 16 question marks, and a dashed box asking for the Bit Recovery Accuracy.
Payload from the lesson's Build It section; flipped positions are not specified by the question.
5. In `safe_tts`, what happens when clone_reference is None?Apply
A call to safe_tts with text and voice only, so clone_reference keeps its default. Below, the four steps verify_consent, tts_model.synthesize, audioseal_embed and c2pa_sign, each marked runs question mark.
Step names from the lesson's safe_tts code; whether each runs is left open.
6. You are launching a voice-cloning product. Which design best covers the lesson's three threats?Design
A product flow: user request, a slot, a cloning model, another slot, then audio out to the public. Below, audio from other tools feeds a third slot. All three slots are question marks.
Schematic; the protections are left open.

Use It

Use caseDefense
Shipping TTS / voice cloningAudioSeal embed on every output (non-negotiable)
Biometric voice unlockAASIST + ECAPA ensemble; liveness challenge
Call-center fraud detectionAASIST on 20% sample of incoming calls
Podcast authenticityC2PA signing on upload, AudioSeal if AI-generated
Research / training detectorsASVspoof 5 train/dev/eval sets
Check: Use It0/4
1. Per the lesson's Use It table, what defense is 'non-negotiable' for shipping TTS or voice cloning, and what's recommended for biometric voice unlock?Apply
A two-row table with use cases shipping TTS or voice cloning, and biometric voice unlock. The defense column is a question mark in both rows.
Use-case labels from the lesson; defenses left open.
2. A call center wants to flag likely voice-cloning fraud without screening every single call, and a podcast network wants to prove which episodes are AI-narrated. What does the lesson's Use It table recommend for each?Apply
Two cards. A call center wants to flag likely voice-cloning fraud without screening every call. A podcast network wants to prove which episodes are AI-narrated. Each has a dashed recommended box left open.
Goals from the question stem; recommendations left open.
3. You must secure a bank's voice unlock against cloned voices, including live ones. Which design fits the lesson best?Design
Two kinds of audio reach a bank's voice unlock: the real customer live and a live voice-cloning tool. A dashed box asks which defense sits at the unlock decision.
Schematic; the defense is left open.
4. A podcast network wants to prove which episodes are AI-narrated and have that proof survive re-uploads. Which approach fits?Design
An AI-narrated episode is uploaded on the network's site, then re-uploaded elsewhere as re-encoded copies. A dashed box asks for a proof of AI narration that survives.
Schematic; the proof is left open.

Pitfalls

  • Watermark without detector ever running. A watermark nobody checks is pointless. Ship the detector in your CI, not just the generator.
  • Detection without calibration. AASIST trained on ASVspoof LA overfits, so real-world accuracy drops once you leave the benchmark. Calibrate on your own domain.
  • Pitch-shift gap. Aggressive pitch shift removes most watermarks outright. Always have a detection fallback.
  • Metadata strip-and-rehost. C2PA is trivially bypassable by re-encoding the file. Pair cryptographic and perceptual (watermark) defenses together, always.
  • Liveness as detection. Asking the user to say a random phrase stops replay attacks. It does nothing against real-time cloning.
Check: Pitfalls0/6
1. Why does the lesson say C2PA metadata signing isn't sufficient on its own?Explain
A file with audio samples and a signed C2PA manifest is reposted by someone else. The file that comes out has audio samples and a dashed box reading manifest question mark.
Schematic; what survives the repost is left open.
2. What do the pitfalls section's 'watermark without detector ever running' and 'detection without calibration' warnings say?Explain
Setup A: a generator embeds a mark and clips are released. Setup B: AASIST trained on ASVspoof LA and deployed on the team's own audio. A closing line asks what goes wrong in each.
Schematic; the problems are left open.
3. What does the pitfalls section say about using a liveness challenge (asking the user to say a random phrase) as detection?Explain
A login shows a prompt that says a random phrase, then a spoken response sent to the login. A dashed box asks what that challenge accomplishes when used as detection.
Schematic; the outcome is left open.
4. A team ships AudioSeal embedding on every output and claims compliance. After an incident they cannot tell whether a leaked clip came from their model. What was missing?Analyze
A team's generator feeds AudioSeal embedding on every output, which are released. A leaked clip of unknown origin goes through a dashed missing-piece box toward the question of whether their model made it.
Schematic of the stem; the missing piece is left open.
5. A team fine-tuned AASIST on ASVspoof 2019 LA and saw near-zero EER on its validation split. Deployed on call audio, accuracy drops sharply. What is the most likely cause?Analyze
AASIST fine-tuned on ASVspoof 2019 LA shows near-zero EER on its validation split, then on the team's own call audio accuracy drops sharply. A dashed box asks for the most likely cause.
Descriptions from the stem; no measured values shown.
6. A bank adds a random-phrase liveness challenge to voice unlock and considers detection unnecessary. An attacker still gets in with a live voice-cloning tool. Why?Analyze
Four steps of a login: the bank shows a random phrase, the attacker feeds it to a live cloning tool, the tool speaks it in the victim's voice, and the voice unlock opens. A dashed box asks why it works.
Steps follow the question stem.

Ship It

Save this as outputs/skill-spoof-defender.md. It picks a detection model, a watermark, a provenance manifest, and an operational playbook for a voice-generation deployment.

Exercises

  1. Easy. Run code/main.py. It runs the toy detector and the toy watermark embed/detect on synthetic audio.
  2. Medium. Install audioseal, embed a 16-bit payload in a TTS output, and re-decode it. Corrupt the audio with noise and measure Bit Recovery Accuracy.
  3. Hard. Fine-tune a RawNet2 or AASIST on ASVspoof 2019 LA and measure EER. Then test it on a held-out set of F5-TTS-generated clips and see how much the out-of-distribution detection degrades.

Key Terms

TermWhat people sayWhat it actually means
ASVspoofThe benchmarkBiennial challenge; 2024 = ASVspoof 5.
EERError rateEqual Error Rate: where false-accept and false-reject rates cross. Lower is better.
CM (countermeasure)DetectorClassifier: real speech vs synthetic / converted.
SASVSpeaker verif + CMIntegrated biometric + spoof detection.
AudioSealMeta watermarkLocalized, 16-bit payload, 485× faster than WavMark.
Bit Recovery AccuracyWatermark survivalFraction of payload bits recovered after attack.
C2PAProvenance manifestCryptographic metadata about creation / authorship.
AASISTDetector familyGraph-attention-based anti-spoofing SOTA.

Further Reading

Code

Ships With