vibevoice-community/

Cortana

This is a fine-tune of the 7B model of VibeVoice. Requires 21.9GB of VRAM for inference (through OpenVoiceLab)

Dataset used for fine-tuning, was 764 audio files available for Halo 1, Halo 2, and Halo 3, via the links below:
Halo 1: https://sounds.spriters-resource.com/xbox/halocombatevolved/asset/413569/
Halo 2: https://sounds.spriters-resource.com/xbox/halo2/asset/436393/?source=genre
Halo 3: https://sounds.spriters-resource.com/xbox_360/halo3/asset/405404/

Fine-tuning parameters used were:

batch_size = 1
drop_rate = 0.2
grad_accum = 1
lr = 2.5e-5
lora_r = 128
lora_alpha = 512
epochs = 20
train_diff = True
bf16 = True
grad_clip = True
max_grad = 0.8
grad_checkpoint = False
diff_weight = 1.4
ce_weight = 0.04
warmup = 0.03
scheduler = "cosine"