On-device Korean verbatim speech-to-text, running entirely in Unity with Sentis (com.unity.ai.inference) 2.6. Hold a button, speak Korean, release, and the clip is transcribed on-device with no server. "Verbatim" means fillers and repeats (어, 음, 그, 막 ...) are kept, not cleaned away.

What it does

Press and hold the Hold to Record button to capture microphone audio (16 kHz).
On release, the recording is transcribed and the Korean text is shown.
Model size (Small / Base / Tiny) is selected in the Inspector (no runtime UI).

Pipeline

Microphone PCM (16 kHz, up to 30 s)
  -> log-mel graph      logmel.sentis        Conv1d/DFT, matches the HF Whisper feature extractor
  -> Whisper encoder    encoder.sentis       Float16
  -> Whisper decoder    decoder(_with_past)  Float16, manual KV cache
  -> Sentis tokenizer   -> Korean text

All three sizes share one log-mel graph. Everything loads from StreamingAssets/ko as .sentis (no raw ONNX at runtime).

Requirements

Unity 6000.5.5f1, com.unity.ai.inference (Sentis) 2.6.1, and a microphone (the OS may prompt for record permission on first use).

Run

Open this project in Unity.
Open Assets/Scenes/CrisperWhisperKo.unity and press Play.
Select the CrisperWhisperKo object in the Hierarchy, pick Model (Small / Base / Tiny) in the Inspector.
Hold the button, speak Korean, release to transcribe. Model can be switched live in the Inspector.

Project layout

Assets/
  Scenes/CrisperWhisperKo.unity     demo scene (UGUI: record button + output/status text)
  Scripts/
    CrisperWhisperKo.cs             mic capture + model select + transcription (manager)
    WhisperRunner.cs             encoder + manual-KV greedy decode
    WhisperText.cs               Sentis tokenizer decode
    WhisperConfig.cs             meta types
  Settings/                      URP asset + renderer
  StreamingAssets/ko/            models the demo loads
    logmel.sentis
    small/  base/  tiny/         encoder + decoder + decoder_with_past (Float16) + meta + tokenizer

The .sentis files are prebuilt (Float16 conversions of the Hugging Face models below). The offline conversion/quantization scripts are not part of this demo project.

Models

The .sentis files under StreamingAssets/ko are Float16 conversions of the CrisperWhisper2 Korean verbatim models. Full model cards, weights (safetensors), evaluation, and transcription examples are on Hugging Face:

Model	Params	Korean CER (lower is better)	Hugging Face repo
small	244M	0.103	Sky-Kim/crisper-whisper2-small-finetuned-ko
base	74M	0.167	Sky-Kim/crisper-whisper2-base-finetuned-ko
tiny	39M	0.216	Sky-Kim/crisper-whisper2-tiny-finetuned-ko

How they were made. small is initialized from CrisperWhisper2.0_small and fine-tuned on Korean verbatim data (KsponSpeech ground truth); base and tiny are soft-KD distilled from that Korean small. This transfers CrisperWhisper's verbatim strength (an English-only model) into Korean. On held-out KsponSpeech, base (74M) beats standard whisper-small (244M, CER 0.171) at one third the size, and all sizes keep 86 to 92% of disfluencies (standard Whisper keeps about 60%).

License: non-commercial / research use only. The models inherit the CrisperWhisper2.0 Non-Commercial Research License (used as init / KD teacher / tokenizer) and the AI Hub KsponSpeech terms (training data). See each model card's NOTICE for the full license chain.

Sky-Kim/crisper-whisper2-unity

CrisperWhisperKo (Unity Sentis demo)

What it does

Pipeline

Requirements

Run

Project layout

Models

Sky-Kim/
crisper-whisper2-unity