Views
No views yet
E + B(A(·)) for both K and V) with a per-token FFN inserted
between the GloVe-attention output H and the α-collapse. A, B are warm-started
from jsanzolac/bpe_glove_300_lora_r300_qwen3/rank_300/checkpoint_final.pt; the FFN is fresh.Linear(300, 300) with GELU, dropout 0.1, residual + post-LayerNorm.InfoNCE(v_S vs [v_T ‖ v_hards], τ=0.05) + 0.1·MSE(v_S, v_T),
with H = 64 mined hard negatives per anchor at batch=256.rank_300/:checkpoint_final.pt — A.weight, B.weight, plus the FFN's l1/l2/l3 weights + LayerNorm.
E is excluded (non-persistent buffer); reinject from jsanzolac/drifting-glove-distilled-r300 at load time.config.jsonvectors_drifted.txt — E + B(A(·)) per vocab row (FFN effect NOT included; it's contextual).train_log.jsonl