MMC Humpback DAC9 Models

Layout

• 10k/
  • SWA+MoE 10k-context checkpoint family
  • includes best_model.pt and resumable checkpoint_step648000.pt
• 32k/
  • Medium SWA+MoE 32k-context checkpoint family
  • includes best_model.pt, resumable checkpoint_step504000.pt, and resumable checkpoint_step506000.pt
  • includes training_log.jsonl for training provenance
• 128k/
  • NSA+MoE 128k-context checkpoint family
  • includes best_model.pt
  • does not currently include a resumable checkpoint_step*.pt

Intended workflows

• scripts/describe_checkpoint.py <checkpoint>
• scripts/train.py ... --init-from <best_model.pt>
• scripts/train.py ... --resume-from <checkpoint_step*.pt> where available
• scripts/generate_dac_9cb_unified.py --checkpoint <checkpoint>

Model families

• 10k/
  • config: audio_large_swa_moe_sanctsound_humpback_dac_9cb_10k.yaml
  • architecture: SWA + MoE
• 32k/
  • config: audio_medium_swa_moe_sanctsound_humpback_dac_9cb_32k.yaml
  • architecture: sliding-window attention + MoE
• 128k/
  • config: audio_medium_nsa_moe_sanctsound_humpback_dac_9cb_128k.yaml
  • architecture: NSA-style compressed attention + MoE

Examples

License and source-data notice