Wind Edge is a custom dense transformer (RMSNorm + RoPE + GQA + SwiGLU) with per-head Q/K normalization. Includes its own WindEdgeForCausalLM / WindEdgeConfig classes — load with trust_remote_code=True.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
m = AutoModelForCausalLM.from_pretrained("arthu1/wind-edge-1.6-sft", trust_remote_code=True)
tok = AutoTokenizer.from_pretrained("arthu1/wind-edge-1.6-sft")

Pipeline

Continued pretraining on GLM reasoning, CodeX thinking, and Nemotron post-training data.
Supervised fine-tuning on Wind Edge SFT, identity pairs, Claude-style assistant data, and safety/chat rows.
Depth pruning (block-influence) to ~0.45B parameters.

North-ML1/Wind-Edge-1.6-Base

Wind Edge 1.6 (SFT)

Architecture

Usage

Pipeline

North-ML1/
Wind-Edge-1.6-Base