Views
No views yet
| Component | Details |
|---|---|
| Framework | SLAM-LLM |
| Speech Encoder | Data2Vec-AQC Hindi (SPRING_INX_data2vec_aqc_Hindi.pt) |
| Encoder Params | 313.27 M (frozen ❄️) |
| Projector | Linear (concat + 2-layer MLP) |
| Projector Params | 15.73 M (trainable 🔥) |
| LLM | google/gemma-3-4b-it |
| LLM Params | 3880.26 M |
| LoRA Trainable Params | 14.90 M 🔥 |
| Total Trainable Params | ~30.63 M |
| Parameter | Value |
|---|---|
| Method | LoRA |
| Rank (r) | 8 |
| Alpha | 32 |
| Dropout | 0.05 |
| Target Modules | q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj |
| Task Type | CAUSAL_LM |
| Parameter | Value |
|---|---|
| Dataset | Hindi 4000 Hours (ESPnet chunked) |
| Train Utterances | 1,924,281 |
| GPUs | 4 × NVIDIA RTX 6000 Ada (48GB each) |
| World Size | 4 |
| Batch Size (per GPU) | 4 |
| Gradient Accumulation | 2 |
| Effective Batch Size | 32 |
| Learning Rate | 1e-4 |
| Warmup Steps | 1000 |
| Optimizer | AdamW (DeepSpeed ZeRO Stage 2, CPU offload) |
| Precision | bf16 |
| Prompt Style | gemma2 |
| Prompt | "Transcribe speech to Hindi text. " |
| Checkpoint | Epoch 1, Step 115000 |
| Test Set | Sentences | WER (%) |
|---|---|---|
| KathBath (clean) | 1929 | 4.79 |
| IndicTTS | 100 | 5.48 |
| KathBath (noisy) | 1929 | 5.70 |
| FLEURS | 418 | 6.52 |
| CommonVoice | 1727 | 7.41 |
| MUCS | 3897 | 7.46 |
pytorch_model.bin).google/gemma-3-4b-itSPRING_INX_data2vec_aqc_Hindi.pt1# Clone SLAM-LLM
2git clone https://github.com/X-LANCE/SLAM-LLM
3
4# Run inference
5python examples/asr_librispeech/inference_asr_batch.py \
6 ++model_config.llm_name=gemma-3-4b-it \
7 ++model_config.llm_dim=2560 \
8 ++model_config.encoder_name=data2vec_aqc \
9 ++model_config.encoder_dim=1024 \
10 ++model_config.encoder_projector=linear \
11 ++model_config.encoder_projector_ds_rate=5 \
12 ++dataset_config.prompt_style=gemma2 \
13 ++ckpt_path=/path/to/pytorch_model.bin1@article{ma2024embarrassingly,
2 title={An Embarrassingly Simple Approach for LLM with Strong ASR Capacity},
3 author={Ma, Ziyang and others},
4 journal={arXiv preprint},
5 year={2024}
6}