Fine-tuned checkpoint for LiquidAI/LFM2.5-Audio-1.5B-JP, trained on Japanese station announcement clips from 3bitquantizers/jp-station-full-v2-vad-trimmed-openai-clip-asr-strict.

The model was trained for Japanese TTS with the system prompt:

Perform TTS in japanese.

Training

Base model: LiquidAI/LFM2.5-Audio-1.5B-JP
Dataset: 3bitquantizers/jp-station-full-v2-vad-trimmed-openai-clip-asr-strict
Training rows: 432
Validation rows: 23
Steps: 1000
Batch size: 2
Learning rate: 3e-5
W&B run: https://wandb.ai/3bitquantizers/Roy_finetuning/runs/eylmj53g

Inference

This checkpoint uses the liquid-audio package.

python

import soundfile as sf
import torch
from liquid_audio import ChatState, LFM2AudioModel, LFM2AudioProcessor

repo_id = "3bitquantizers/LFM2.5-Audio-1.5B-JP-jp-station-v2-vad-trimmed"
prompt = "次は、渋谷、渋谷です。お出口は右側です。"

processor = LFM2AudioProcessor.from_pretrained(repo_id, device="cuda").eval()
model = LFM2AudioModel.from_pretrained(repo_id, device="cuda", dtype=torch.bfloat16).eval()

chat = ChatState(processor)
chat.new_turn("system")
chat.add_text("Perform TTS in japanese.")
chat.end_turn()
chat.new_turn("user")
chat.add_text(prompt)
chat.end_turn()
chat.new_turn("assistant")

frames = []
with torch.no_grad():
    for token in model.generate_sequential(
        **chat,
        max_new_tokens=512,
        text_temperature=None,
        text_top_k=1,
        audio_temperature=0.8,
        audio_top_k=4,
    ):
        if token.numel() == 1:
            continue
        if token[0].item() == 2048:
            break
        frames.append(token.detach())

codes = torch.stack(frames, dim=1).to(device="cuda", dtype=torch.long)
wav = processor.mimi.eval().decode(codes.unsqueeze(0))[0].float().cpu().squeeze()
sf.write("sample.wav", wav.numpy(), 24_000)

Notes

This is a narrow-domain fine-tune for Japanese station-announcement style speech. It is not intended as a general-purpose speech model.