CrisperWhisper 2.0

The most accurate verbatim speech recognition you can run in production:
controllable, multilingual, and timed to the word.

Release post ·
Paper ·
Full documentation ·
Models ·
Benchmark

<a href="https://www.nyra-labs.com/crisperwhisper">
<img src="https://img.shields.io/badge/%E2%96%B6%20%20TRY%20IT%20NOW-2563eb?style=for-the-badge" alt="Try it now">
</a>

Most speech-to-text systems never actually decide whether to write down what
was said or what was meant. They inherit that choice from their training
data and apply it inconsistently. CrisperWhisper 2.0 makes it an explicit,
controllable choice. One recording, two transcripts:

Verbatim, exactly what was said, in one consistent format:
[um] so we we need to, to reschedule the th- thursday meeting to [uh] march third at nine thirty [laughter]

Intended, the clean version the speaker meant, with numbers, dates,
and emails formatted the way you'd write them:
So we need to reschedule the Thursday meeting to March 3 at 9:30.

On top of that:

• Word-level timings. Around 30 ms mean boundary error on read speech
  and 41 ms on conversational speech, the most precise word timing of any
  system we benchmarked, on both.
• Verbatimize. Upgrade transcripts you already have: given audio plus a
  trusted clean transcript, the model reproduces your content word-for-word
  and inserts only the disfluencies and vocal events actually present in the
  audio (rare-word recall jumps from 6.8% to 96.1% vs. re-transcribing).
  This turns the world's abundant clean corpora into verbatim ones, ready
  for TTS data, clinical speech analysis, and dataset construction.
• Multilingual. Verbatim and intended modes work across most languages
  Whisper supports. CrisperWhisper 2.0 tops the
  Nyra Verbatim Speech Benchmark
  leaderboard for disfluency F1 across ten languages, ahead of every
  closed-source alternative we tested.
• Seamless longform. Audio of any length, transcribed without the usual
  chunk-boundary artifacts: each window continues from the words already
  transcribed (conditional continuation), so there are no duplicated or
  dropped words at the seams and no fragile timestamp-token bookkeeping.
• Production inference. A CTranslate2 runtime with speculative decoding
  and built-in mitigation of Whisper's looping-hallucination failure mode.

Performance

The Nyra Verbatim Speech Benchmark
scores fillers, repetitions, cut-offs, and vocal sounds as separate, typed
metrics. Its headline number is disfluency F1: how reliably a system
writes down the disfluencies that were actually spoken, without inventing
ones that weren't. Averaged over ten languages:

_{* CrisperWhisper 1.0 is English/German-only; its average covers those
two languages. English and German use human-labeled evaluation sets; the
other eight languages use synthetic verbatim sets. Per-language breakdowns
and how the metric is computed are in the
benchmark post.}

Word-timing accuracy

Mean absolute word-boundary error on read speech (TIMIT), lower is better:

_{Scored on exactly the words each system gets right. How the timings
are extracted from supervised cross-attention, plus results on
conversational speech, are in the
aligner post.}

Install

# NVIDIA GPU (Linux): fastest, includes speculative decoding.
# An NVIDIA driver is all you need; CUDA libraries arrive via pip.
pip install "crisperwhisper[ct2]"

# Pure PyTorch: runs anywhere torch does (macOS, Windows, CPU)
pip install "crisperwhisper[transformers]"

Quickstart

from crisperwhisper import CrisperWhisperModel

model = CrisperWhisperModel()          # nyralabs/CrisperWhisper2.0_large
# or pick a size: CrisperWhisperModel("turbo")  # turbo / medium / small

# Verbatim transcription (default): every filler, repetition, stutter,
# false start, and vocal event
result = model.transcribe("meeting.wav", language="en")
print(result.text)

# Intended: the clean, readable version
clean = model.transcribe("meeting.wav", language="en", mode="intended")

# Word-level timestamps
result = model.transcribe("meeting.wav", language="en", word_timestamps=True)
for w in result.words:
    print(f"{w.start:6.2f}-{w.end:6.2f}  {w.word}")

# Verbatimize: upgrade an existing clean transcript with the
# disfluencies that are actually in the audio
result = model.verbatimize("clip.wav", "I think we should ship it Friday.")

Audio longer than 30 seconds is handled automatically (see
longform below). The first load of a model
downloads it from HuggingFace and, on the ct2 backend, converts it once
into a local cache. That one-time conversion needs torch and
transformers, which the lean [ct2] extra does not install -- add them
once with:

pip install "crisperwhisper[ct2,convert]"   # or add [convert] later

A CPU-only torch is enough for the conversion, and the dependencies can
be uninstalled once the converted model is cached. Loading a directory
that already contains a converted CT2 model (model.bin) never needs
them.

Models

The standard models are released under a
non-commercial research license
and are available for commercial licensing. The Pro models are available
under commercial license only. For both,
get in touch.

Faster inference: speculative decoding (ct2)

A small draft model proposes tokens and the main model verifies them. Same
output, 1.3 to 1.4x faster:

model = CrisperWhisperModel("large", draft_model="turbo")
result = model.transcribe("meeting.wav", language="en",
                          speculative_decoding=True)

What else is in the box

Everything below works out of the box and is covered in depth in
DOCS.md:

How it works

Each mechanism has a deep-dive post:

• Measuring verbatimness: the Nyra Verbatim Speech Benchmark.
  Typed metrics for fillers, repetitions, cut-offs, and vocal sounds instead
  of one opaque WER number.
• How we unlocked multilingual style-controlled transcription at scale.
  Verbatim/intended control across languages.
• Turning emergent cross-attention into a precise aligner.
  Supervising Whisper's alignment heads for word boundaries around 30 ms.
• Longform transcription with conditional continuation.
  Resolving window seams with the words already transcribed instead of
  fragile timestamp tokens.
• Closing the verbatim data gap with Verbatimize.
  Upgrading clean corpora into verbatim ones at scale.
• Faster inference and mitigating hallucinations.
  The CTranslate2 stack, speculative decoding, and the anti-looping decoder.

Documentation

DOCS.md covers the full API: backends and their trade-offs,
every transcribe() option, dual-mode transcription, forced alignment,
longform strategies, speculative-K tuning, hallucination-repair thresholds,
quantization, model conversion, and the result object.

License

The inference code in this repository is MIT-licensed (see
LICENSE): use it freely, commercially or otherwise.
crisperwhisper/features.py is vendored from
faster-whisper (MIT, SYSTRAN).

The model weights are not MIT. They are released under the
Nyra Health Non-Commercial Research License:
free for research and other non-commercial use; any commercial use requires
a commercial license. The Pro models are available under commercial license
only. For commercial licensing of either,
contact Nyra.