Base Models:
- ZeroXClem/Qwen2.5-7B-HomerCreative-Mix
- ZeroXClem/Qwen2.5-7B-HomerAnvita-NerdMix
Merge Method: passthrough
Note: This is a frankenmerge model with modified architecture

Configuration

yaml

slices:
  - sources:
    - model: ZeroXClem/Qwen2.5-7B-HomerCreative-Mix
      layer_range: [0, 28]
  - sources:
    - model: ZeroXClem/Qwen2.5-7B-HomerAnvita-NerdMix
      layer_range: [0, 28]
merge_method: passthrough
dtype: bfloat16

Usage

This model can be used with the standard transformers library:

python

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("Ro-xe/FMixIA-FrankenMerge-9.5B-PT-9")
tokenizer = AutoTokenizer.from_pretrained("Ro-xe/FMixIA-FrankenMerge-9.5B-PT-9")