A quick DPO pass, done with several random seeds and taking the mean. Included data from rewriting low-scoring RLVR turn samples with judge feedback, as chosen vs rejected originals.

Warning: On double checking, the turn samples were rewritten by Gemma 4 31B - this and 4.3.2 thus have some Gemma influence.

arsenic-v4.3-dpo-karcher

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method.

Models Merged

The following models were included in the merge:

/home/ann/Documents/Projects/training_models/outputs/baked_v43_seed2
/home/ann/Documents/Projects/training_models/outputs/baked_v43_seed1
/home/ann/Documents/Projects/training_models/outputs/baked_v43_seed5
/home/ann/Documents/Projects/training_models/outputs/baked_v43_seed3
/home/ann/Documents/Projects/training_models/outputs/baked_v43_seed4

Configuration

The following YAML configuration was used to produce this model:

yaml

models:
  - model: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed1
  - model: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed2
  - model: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed3
  - model: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed4
  - model: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed5
merge_method: karcher
parameters:
  normalize: true
  int8_mask: true
tokenizer_source: /home/ann/Documents/Projects/training_models/outputs/baked_v43_seed1
dtype: bfloat16

Lambent/Arsenic-Shahrazad-12B-v4.3.1

arsenic-v4.3-dpo-karcher

Merge Details

Merge Method

Models Merged

Configuration

Lambent/
Arsenic-Shahrazad-12B-v4.3.1