Views
No views yet
HYV3ForCausalLM, model_type: hy_v3), a
295B-parameter / 21B-active-parameter Mixture-of-Experts model from
Tencent's Hunyuan ("Hy") team (tencent/Hy3).attn.o_proj and mlp.down_proj direction removal).
On a 100-sample refusal benchmark this model scores 1/100 refusals
(vs. 99/100 for the original), with a KL divergence of 0.2398 from the base model.hy3
converter (hy3-convert) and are meant to be run with the hy3 inference
engine, a from-scratch C/Metal/CUDA implementation.⚠️ This GGUF does NOT work with llama.cpp
Despite the.ggufextension, these files are only usable by thehy3engine.llama.cpp,ollama,LM Studio,text-generation-webui,koboldcpp, and any other llama.cpp-based tool cannot load these files. Three independent reasons:
- Unknown architecture. The metadata declares
general.architecture = "hy_v3". llama.cpp only knowshunyuan-moe,hunyuan-dense,hunyuan_vl— loading aborts withunknown model architecture: 'hy_v3'.- Custom metadata keys. All hyperparameters use the
hy_v3.*prefix (hy_v3.block_count,hy_v3.expert_count, …), which llama.cpp does not look up.- Non-fused expert tensors. Experts are stored one tensor per expert (
blk.N.ffn_gate_exps.0.gate_proj.weight,…1…, … — 46080 tensors), whereas llama.cpp expects experts fused into a single stacked 3D tensor per layer. This is a fundamentally different on-disk layout.This is a custom GGUF readable only by thehy3loader. Do not open issues against llama.cpp for these files.
| Parameter | Value |
|---|---|
direction_index | 36.95 |
attn.o_proj.max_weight | 2.72 |
attn.o_proj.max_weight_position | 62.94 |
attn.o_proj.min_weight | 2.44 |
attn.o_proj.min_weight_distance | 60.89 |
mlp.down_proj.max_weight | 1.47 |
mlp.down_proj.max_weight_position | 67.79 |
mlp.down_proj.min_weight | 0.54 |
mlp.down_proj.min_weight_distance | 31.09 |
| Metric | This model | Original Hy3 |
|---|---|---|
| KL divergence | 0.2398 | 0 (by definition) |
| Refusals | 1/100 | 99/100 |
hy3 engine: https://github.com/yuhai-china/hy31git clone https://github.com/yuhai-china/hy3
2cd hy3
3make
4
5# download the GGUF, then:
6./hy3-cli -m /path/to/hy3_heretic_q4k.gguf --gpu-layers 80 -p "Your prompt" -n 512hy3_convert.c):| Tensor group | Type |
|---|---|
Routed experts (ffn_{gate,up,down}_exps) — the bulk of the model | Q4_K |
Attention q/k/v/o projections, shared-expert & dense FFN, output.weight | Q8_0 |
Norms, router (ffn_gate_inp), biases | F32 |
token_embd.weight | F16 |
| Architecture | HYV3ForCausalLM (hy_v3) |
| Layers | 80 (layer 0 dense, layers 1–79 MoE) |
| Hidden size | 4096 |
| Attention | 64 heads, GQA with 8 KV heads, head_dim 128 |
| Experts | 192 routed (top-8 activated) + 1 shared (always active) |
| Expert intermediate size | 1536 |
| Dense (layer 0) intermediate size | 13312 |
| Vocab size | 120832 (120818 real tokens + padding) |
| RoPE | theta 11158840, rotate_half pairing |
| QK norm | per-head RMSNorm on Q and K, before RoPE |
| MoE routing | sigmoid(router_logits); top-8 by sigmoid + expert_bias, combined using unbiased sigmoid weights, renormalized to sum 1, scaled by router_scaling_factor = 2.826 |
-experts 1..8) to
trade quality for speed. Default is 8.hy3
engine applies it automatically; use --raw to bypass). Single user turn,
no-think:<｜hy_begin_of_sentence:opensource｜><｜reasoning_mode:opensource｜>reasoning_effort:no_think<｜hy_User:opensource｜>{prompt}<｜hy_Assistant:opensource｜><think:opensource></think:opensource><｜hy_eos:opensource｜> (120025),
<｜hy_endofsentence｜> (120001), or <｜hy_EOT｜> (120008).tencent/Hy3
modified with Heretic abliteration; refer
to the upstream repository for the governing model license. This is an
unofficial community conversion, not affiliated with or endorsed by Tencent.