You can deploy this model and start earning money today!

MohammadKhosravi/
llama3.1-8b-cefr-steering-1layer-head-ordinal-universal

Name: llama3.1-8b-cefr-steering-1layer-head-ordinal-universal
Availability: InStock

ordinal-classification

controllable-text-generation

pplm

Views

No views yet

Llama-3.1-8B CEFR 1-Layer MLP Steering Head (Ordinal Universal)

This repository contains a 1-Layer Linear Classifier Head designed to steer the hidden representations of meta-llama/Llama-3.1-8B-Instruct towards specific CEFR proficiency levels (A1-C2).

🧠 Architectural Rationale

Unlike deeper multi-layer MLPs, this 1-layer head maps hidden representations ($d=4096$) directly to CEFR logits ($d=6$) using a single linear layer with an input dropout of 0.35. By eliminating ReLU activations and LayerNorm scaling, this architecture ensures:

Strict Gradient Continuity: Eliminates vanishing/exploding gradient boundaries.
Stable Backpropagation Landscape: The continuous linear gradient vector provides stable, non-saturating paths for real-time decoding-time token perturbation (PPLM).

📊 Training Specifications & Performance

Loss Function: Custom Ordinal Weighted Loss (Weighted Cross-Entropy + MSE Ordinal Distance Penalty)
Optimizer: AdamW (Learning Rate: 3e-4, Weight Decay: 0.01)
Epochs: 50
Batch Size: 128
Peak Validation Strict Accuracy: 85.16%
Peak Validation Adjacent Accuracy: 97.32%

MohammadKhosravi/llama3.1-8b-cefr-steering-1layer-head-ordinal-universal

Llama-3.1-8B CEFR 1-Layer MLP Steering Head (Ordinal Universal)

🧠 Architectural Rationale

📊 Training Specifications & Performance

MohammadKhosravi/
llama3.1-8b-cefr-steering-1layer-head-ordinal-universal