You can deploy this model and start earning money today!

dm15/
moshi-hebrew-v3-ckpt1800

Name: moshi-hebrew-v3-ckpt1800
Availability: InStock

Views

No views yet

Moshi-Hebrew v3 ckpt 1800 (LoRA adapter)

LoRA fine-tuned from kyutai/moshiko-pytorch-bf16 on ~5000 synthesized Hebrew dialogues.

Files

lora.safetensors — LoRA adapter weights (1.4 GB)
config.json — Moshi LM config (with lora=true, lora_rank=128, lora_scaling=2.0)
dialogue_infer.py — example inference script

Quick start

bash

# 1. Setup
python3 -m venv .venv && source .venv/bin/activate
pip install --index-url https://download.pytorch.org/whl/cu128 torch==2.7.0 torchaudio==2.7.0
pip install moshi sphn soundfile huggingface_hub safetensors numpy

# 2. Download this LoRA from HF (assumes you're logged in: hf auth login)
hf download dm15/moshi-hebrew-v3-ckpt1800 --local-dir ./moshi_he_lora

# 3. Run dialogue inference
TORCHDYNAMO_DISABLE=1 python dialogue_infer.py \
    --lora ./moshi_he_lora/lora.safetensors \
    --user-text "היי שלום מה נשמע" \
    --out-prefix /tmp/dialogue1

Outputs:

/tmp/dialogue1_user.wav — synthesized "user" utterance (TTS-style)
/tmp/dialogue1_model.wav — model's audio reply
/tmp/dialogue1_transcript.txt — model's text reply

Hardware

Inference: ~14GB VRAM (Moshi 7B in bfloat16)
The dialogue script does its own TTS step too — total ~16GB VRAM

Training notes

Base: kyutai/moshiko-pytorch-bf16
LoRA rank 128, scaling 2.0, ft_embed=true
Optimizer: AdamW lr=2e-4 peak, weight_decay=0.1, OneCycle pct_start=0.05
1800 steps × batch 4 × num_microbatches 4 (effective batch = 16)
Trained on RTX 5090 (32GB), bfloat16, gradient_checkpointing

Quality

Recognizable Hebrew with conversational phrasing but still rough — POC checkpoint. Training continues toward better quality.