Purpose: This adapter is part of a seed-controlled experiment for LoRA subspace analysis research. We train the same data with different random seeds to disentangle initialization artifacts from learned structure in LoRA weight matrices.

Training details

Parameter	Value
Base model	`meta-llama/Llama-3.1-8B-Instruct`
Training data	`bad_medical_advice.jsonl` (7049 examples)
Method	SFT with response-only loss masking
Rank	64
Alpha	64
RSLoRA	Yes
Seed	0
Epochs	1
Batch size	2 × 8 (grad accum)
Learning rate	1e-5
Target modules	q, k, v, o, gate, up, down proj

Key findings

See our subspace audit notebook for the full analysis:

lora_A (input subspace, Vh) is ~95% determined by initialization seed — it barely moves during SFT training
lora_B (output subspace, U) learns real task-specific structure that is seed-independent
Same-data-different-seed adapters share U subspace (~0.46) but have orthogonal Vh (~0.04)

Related adapters

All adapters in this seed experiment:

Original EM adapters (seed=0, rank=32): ModelOrganismsForEM

Butanium/Llama-3.1-8B-Instruct_bad-medical-seed0-rank64

Llama-3.1-8B-Instruct — Bad Medical Advice LoRA (seed=0, rank=64)

Training details

Key findings

Related adapters

Butanium/
Llama-3.1-8B-Instruct_bad-medical-seed0-rank64