Recovery adapter and quantized GGUF for the REAP-20B base model. The adapter is a light rough-edges pass, not a domain rewrite. The GGUF is smoke-tested only. Validate on your own workload.

Nemotron 3.5 Lightning REAP-20B — LoRA adapter + IQ4_NL GGUF

Two artifacts for the REAP-20B base model (20B-total / 3B-active, general-calibrated expert pruning of Nemotron 3.5 Lightning):

adapter/ — a light recovery LoRA (rank 8, ~1 epoch) trained on complete rows from a mix of subagent-orchestration and general/coding/math data, deliberately light: it clears small rough edges from the pruning, not teach new behavior or strict output formats.
Nemotron-3.5-Lightning-30B-A3B-REAP-20B-IQ4_NL.gguf — the base + adapter merged and quantized to IQ4_NL (11.5 GB), for llama.cpp / LM Studio on 24GB-class laptops.

Using the adapter

Apply it on top of the base repo's BF16 weights:

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "sleepyeldrazi/Nemotron-3.5-Lightning-30B-A3B-REAP-20B",
    torch_dtype=torch.bfloat16, trust_remote_code=True)
model = PeftModel.from_pretrained(model, "sleepyeldrazi/Nemotron-3.5-Lightning-30B-A3B-REAP-20B-LoRA-IQ4NL/adapter")
# optionally merge: model = model.merge_and_unload()

PEFT version note: trained with PEFT 0.20.0. Unpatched 0.20.x refuses mixer.out_proj as a LoRA target on nemotron_h (a Mamba-architecture guard). Either use PEFT ≤ 0.18.1, or remove "nemotron_h" from the mamba_model_types set in peft/tuners/tuners_utils.py (safe for this hybrid — its in_proj/out_proj are plain nn.Linear layers).

Adapter details


Rank / alpha	8 / 16, dropout 0.05
Training data	329 complete rows (content ≥300 chars, terminal punctuation): 140 subagent/orchestration + 189 general/coding/math, generated by the full model
Recipe	PEFT, bf16, lr 3e-5, ~1 epoch, seqlen 4096, content-first windowing
Targets	attention q/k/v/o, Mamba in/out projections, shared experts (routed experts are fused 3D params — not PEFT-LoRA-able; see the base repo's `lora/` notes)
Purpose	rough-edges cleanup after expert pruning — light by design

Using the GGUF

bash

# llama.cpp ≥ b10326 (nemotron_h_moe support)
llama-server -m Nemotron-3.5-Lightning-30B-A3B-REAP-20B-IQ4_NL.gguf -c 131072 --reasoning-parser nemotron_v3

Context length: -c 131072 (128K) is a safe default for 16GB-VRAM GPUs and 24GB Macs. KV is only ~6KB/token (6 attention layers, 2 KV heads), so 128K context is roughly 0.8GB and even 256K is ~1.6GB. The model supports up to 1M tokens (256K is NVIDIA's single-GPU deployment length) — on laptops the practical limit at longer contexts is prefill time, not memory.

Base model + full methodology: Nemotron-3.5-Lightning-30B-A3B-REAP-20B
Base: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16

License

OpenMDW License Agreement, version 1.1 (same as the base model). See LICENSE.

sleepyeldrazi/Nemotron-3.5-Lightning-30B-A3B-REAP-20B-LoRA-IQ4NL

Nemotron 3.5 Lightning REAP-20B — LoRA adapter + IQ4_NL GGUF

Using the adapter

Adapter details

Using the GGUF

Related

License

sleepyeldrazi/
Nemotron-3.5-Lightning-30B-A3B-REAP-20B-LoRA-IQ4NL