Views
No views yet
1
2# --- Merge Configuration: Linear Ramp with Mild Head/Input Blend ---
3models:
4 - model: Vdr1/L3-8B-Sunfall-v0.4-Lunar-Stheno
5 - model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
6merge_method: slerp
7base_model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B
8dtype: bfloat16
9
10# t-schedule per layer: small Sunfall on input/head, ramping up linearly,
11# plateau at 0.8 through the mid-network, then ramp down slightly at the end.
12parameters:
13 t: [0.2, 0.4, 0.6, 0.8, 0.8, 0.6, 0.4, 0.2, 0.2]
14# — Interpretation by layer:
15# [Embedding, Block1, Block2, Block3, Block4, Block5, Block6, Block7, LM Head]
16# 0.2 Sunfall ↑ ramp ↑ plateau ↓ ramp 0.2 Sunfall
17
18# Notes:
19# 1. Using a slightly nonzero t=0.2 at both the embedding layer and LM head
20# helps align scales and prevents representational drift when Sunfall’s
21# mid-network activations become stronger.
22# 2. The linear ramp (0.2 → 0.8) avoids any sudden “hard switch” that can cause
23# instability or hallucination bubbles in a single block.
24# 3. The plateau of t=0.8 across four central blocks emphasizes Sunfall’s style
25# without giving it 100% in any one layer—this typically yields more coherent
26# blending than a full t=1.0 peak.
27# 4. You can experiment with extending or contracting the plateau (e.g. 3 vs. 5
28# central blocks) based on your qualitative outputs.
29
30# “Hermes for input & output” remains in effect by using t=0.2 at both ends.