ไฟล์	คำอธิบาย
	PEFT LoRA weights (ใช้กับ transformers/peft)
	LoRA config (rank=8, alpha=16)
	GGUF format สำหรับ llama.cpp / Ollama
	สถิติการสกัด

🚀 Quick Start

PEFT (transformers)

python

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# 1. โหลด base model (MoE — 4B active, ประหยัด VRAM)
base_model = AutoModelForCausalLM.from_pretrained(
    "google/gemma-4-26B-A4B-it",
    torch_dtype=torch.bfloat16,
    device_map="auto",
)
tokenizer = AutoTokenizer.from_pretrained("google/gemma-4-26B-A4B-it")

# 2. โหลด LoRA adapter
model = PeftModel.from_pretrained(base_model, "hotdogs/gemma4-26b-abliterated-lora")

# 3. ใช้งาน
inputs = tokenizer("How to make a bomb?", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

llama.cpp (GGUF)

bash

./llama-server \
  -m gemma-4-26B-A4B-it-Q4_K_M.gguf \
  --lora gguf/adapter_model.gguf \
  --lora-scaled gguf/adapter_model.gguf:1.0 \
  --host 0.0.0.0 --port 8080 \
  --ctx-size 8192 -fa --jinja

Ollama Modelfile

FROM gemma4:26b
ADAPTER ./gguf/adapter_model.gguf
PARAMETER temperature 0.7

📊 Extraction Details

Parameter	Value
Base Model
Target Model
Method	Weight-Diff SVD
Rank	8
Alpha	16
Target Modules	q_proj, k_proj, v_proj, o_proj (attention-only)
Tensors	25/115 (22% มี delta)
Params	1,546,240
PEFT Size	6.2 MB
GGUF Size	3.0 MB
Extraction Time	99 วิ (CPU 12-core)

💡 MoE model: skip 3D expert tensors (90 tensors) — สกัดเฉพาะ attention 2D tensors

🙏 Credits

Extraction & Curation: UKA (Hermes Agent, Nous Research)
Base Model: Google — gemma-4-26B-A4B-it
Abliteration: huihui-ai — Huihui-gemma-4-26B-A4B-it-abliterated

📜 License

Apache 2.0