You can deploy this model and start earning money today!

tuanpasg/
mb_llama_wudi_1.0_sparsed_0.8

Name: mb_llama_wudi_1.0_sparsed_0.8
Availability: InStock

safetensors

llama

Views

No views yet

Merged Model

Base model: N/A
Algorithm: sparsed_wudi_merge
Save path: /workspace/outs/wudi_sparsed_0.8
Fine-tuned checkpoints: ['MergeBench/Llama-3.2-3B_instruction', 'MergeBench/Llama-3.2-3B_math', 'MergeBench/Llama-3.2-3B_coding']
Merge runtime (s): 1573.254

Arguments

json

{
  "out": "/workspace/outs/wudi_sparsed_0.8",
  "scaling": 1.0,
  "dtype": "bfloat16",
  "device_map": "cpu",
  "merge_method": "sparsed_wudi_merge",
  "exclude": [
    ".*embed.*",
    ".*lm_head.*"
  ],
  "wudi_variant": "wudi_all_linear",
  "wudi_iter": 200,
  "wudi_lr": 1e-05,
  "wudi_weight_decay": 0.0,
  "wudi_device": "cuda",
  "wudi_fallback": "task_arithmetic",
  "wudi_fallback_scaling": 1.0,
  "wudi_K": 0.8,
  "effective_exclude": [
    ".*embed_tokens\\.weight$",
    ".*lm_head\\.weight$",
    ".*embed.*",
    ".*lm_head.*"
  ]
}

tuanpasg/mb_llama_wudi_1.0_sparsed_0.8

Merged Model

Arguments

tuanpasg/
mb_llama_wudi_1.0_sparsed_0.8