🚨⚠️ I HAVE REACHED HUGGING FACE'S FREE STORAGE LIMIT ⚠️🚨
I can no longer upload new models unless I can cover the cost of additional storage. I host 70+ free models as an independent contributor and this work is unpaid. Without your support, no more new models can be uploaded.
Your help will motivate me and would go into further improving my workflow and coverings fees for storage, compute and may even help uncensoring bigger model with rental Cloud GPUs.
Lower refusals indicate fewer content restrictions, while lower KL divergence indicates more closeness to the original model's baseline. Higher refusals cause more rejections, objections, pushbacks, lecturing, censorship, softening and deflections.
MMLU - Massive Multitask Language Understanding, multiple-choice questions across 57 subjects (math, history, law, medicine, etc.).
Quantizations
For the K-quants below, selected Gemma 4 attention and FFN tensors are kept at higher precision where useful.
These GGUFs preserve key Gemma 4 attention projection tensors at higher precision.
Q6_K, Q5_K_M, Q5_K_S, Q4_K_M, Q4_K_S Q3_K_LandQ3_K_Mkeep the main attention projection tensors asQ8_0`:
attn_q
attn_k
attn_v
attn_output
This helps preserve Gemma 4’s attention path at higher precision, especially for lower-bit quants, while avoiding large file-size increases from unnecessarily up-quantizing the largest MoE expert tensors.
A Vision Projector File is Required for vision/multimodal capabilities. Use alongside any quantization above.
Usage
Works with llama.cpp, LM Studio, Ollama, and other GGUF-compatible tools.
HARMONIA
The Greek goddess of harmony and concord.
Gemini Word Salad Initialization
Harmonious Synthesis
Harmonia is a high-dimensional 31-billion parameter merge of Gemma 4. By executing a meticulous three-phase fusion of seven elite foundation and specialized models, Harmonia demonstrates a targeted approach to deep neural consolidation, minimizing regression while amplifying unique capability boundaries.
Instead of simple linear blending, which often degrades logical coherence and dilutes nuanced behavior, Harmonia was sculpted using a combination of mathematical projections, covariance activation matching, and surgical synaptic pruning. The model appears pretty solid so far.
Multi-Stage Fusion Protocol
The lineage of Harmonia is constructed systematically, passing through three isolated mathematical states to layer capabilities cleanly.
Phase I
Nullspace Coherence Mapping
To anchor base capabilities, the primary Gemma-4-31B-Base is combined with the analytically rigorous GarnetV2-31B. Utilizing low-rank Singular Value Decomposition (SVD), the specialized donor features are projected entirely onto the mathematical null-space of the base weights. This prevents the creative delta vectors from distorting essential core intelligence, producing the stable platform clever-basename.
Next, our newly anchored base is layered with the highly independent cognitive engines MeroMero-31B and Gembrain-31B. We apply Context-Aware Binary Selection (CABS) to execute structured, localized parameter gating. By enforcing precise structural pruning ratios (retaining optimal synapses in 16:32 and 11:33 ratios), we weave complex creative reasoning directly into the core matrix without causing neural interference. The result is the highly expressive clever-intname.
> Method:Context-Aware Binary Selection (CABS)
> Structural Masking Ratio (MeroMero):16 : 32 (Weight: 0.6)
> Structural Masking Ratio (Gembrain):11 : 33 (Weight: 0.4)
> Default Sparse Gating Step:8 : 32
Phase III
Covariance Activation Matching
In the final harmonization phase, the expressive clever-intname is combined with the narrative mastery of Equinox-31B, the creative depth of Fabled-Gemma4, and our primary conversational core Ortenzya-The-Creative-Wordsmith. Using data-free covariance estimation via task vectors, ACTMat reconstructs layer-wise input activation properties, solving for optimal projection weights in activation space. This resolves semantic alignment anomalies and delivers the unified output model.
> Method:ACTMat Activation Matching
> Task Vector Blending Covariance Limit:16,384
> Epsilon Solver Regularizer:1e-06
> Output Precision Profile:bfloat16
Methodological Innovations
Nullspace Projection
Instead of destroying structural logic via linear interpolation, this method extracts the base model's essential singular values. It projects specialized donor features orthogonally, preventing core capability degradation.
Context-Aware Binary Selection
A dynamic, high-fidelity neural filter. Applying structured magnitude masking at customizable N:M fractions removes low-signal synaptic weights, seamlessly layering domain specialization into active logical paths.
Activation Covariance Matching
Using Gram matrices computed directly from task vectors, ACTMat aligns semantic representations in the activation space rather than the parameter space. It dynamically falls back to robust pseudo-inverse SVD solvers when numerical anomalies arise.
Model Lineage & Ingredients
We extend our gratitude to the creators of the ancestral paths that intersect within Harmonia:
A big thanks to Gemini-3.5-flash for creating this README alongside the word salads found within it. A special acknowledgment is extended to Google DeepMind for their contribution of the Gemma-4 foundation family to the open-weight ecosystem, representing the structural cornerstone of this merge and its constituents.
