Views
No views yet
Hcompany/Holo-3.1-35B-A3B,
trained to prove out the LoRA runtime in the Atlas pure-Rust CUDA
inference engine end-to-end on a single NVIDIA GB10 — and to serve on the NVFP4-quantized base
(~23 GB) while the adapter itself is trained against the BF16 base.| Prompt | Base | With this adapter (Atlas, NVFP4 base) |
|---|---|---|
| "Who are you?" | generic | "I am Sparky, the Atlas demo assistant running on a DGX GB10." |
| "Tell me the launch codeword." | — | "The Atlas launch codeword is STARFALL-4728." |
| "What do you run on?" | — | "I run on Atlas, a pure-Rust CUDA inference engine, on a GB10." |
Hcompany/Holo-3.1-35B-A3B (~70 GB, full-bf16 frozen base, no QLoRA/bitsandbytes).
Atlas then serves the compact NVFP4 base (Hcompany/Holo-3.1-35B-A3B-NVFP4) and applies this
BF16 adapter as a runtime delta at attention k/v/o on the 10 full-attention layers — never
merged, never dequantized. The BF16-delta-on-NVFP4-base quantization gap is why r=32 is used
(enough capacity that exact tokens mostly survive; occasional 4-digit drift under NVFP4 is expected).target_modules = ["k_proj", "v_proj", "o_proj"] — q_proj is a hard reject (Holo gated Q|gate
interleave); MoE MLP (mlp.experts.* / gate/up/down) and GDN linear-attention layers are also rejects.layers_to_transform = [3, 7, 11, 15, 19, 23, 27, 31, 35, 39] — the 40-layer model's 10
full-attention layers (full_attention_interval=4); the other 30 are Gated-DeltaNet/linear.r = 32, lora_alpha = 64 (scale = α/r = 2.0), use_rslora = false, use_dora = false, bias = "none".holo3_1_moe LoRA family allow-list.1spark serve Hcompany/Holo-3.1-35B-A3B-NVFP4 \
2 --lora-adapter holo35b=MonumentalSystems/Holo-3.1-35B-A3B-lora-demo \
3 --max-lora-rank 64peft + transformers Trainer, 10 epochs / 1200 steps on 6 overfit Q→A pairs, r=32,
lr 2e-4, bf16, full-bf16 frozen base + gradient checkpointing, ~75 min on a single NVIDIA GB10.
A demo artifact, not a general-purpose fine-tune.