Abliterated: Orthogonal refusal projection removed from all linear layers
MoE 2/16: 2 active experts per token, 16 total
GLA: Gated Linear Attention — O(1) recurrent state, infinite context capability
Partial RoPE (50%) + YaRN 8x scaling
Uncensored: No refusal, no guardrails, no alignment filtering

Details

Param	Value
Parameters	~12.8B total, ~8B active
Layers	80
Hidden	2048
Heads	8 × 128d
Experts	16 (top-2)
Vocab	262K
Context	128K native, 1M with YaRN

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
model = AutoModelForCausalLM.from_pretrained("BeheraBoi/yasha-8b-abliterated")
tokenizer = AutoTokenizer.from_pretrained("BeheraBoi/yasha-8b-abliterated")