Nemotron 3 Super 120B-A12B — Cybersecurity Theory LoRA (MLX)
LoRA adapter for NVIDIA Nemotron 3 Super (120B total / ~12B active) trained for cybersecurity domain knowledge: offensive methodology, defensive concepts, CTF reasoning, vulnerability catalogs, and security tooling literacy.
This is the Stage-1 theory specialist from the Mati MultiLoRA coordinator stack. It is intended for instruction-following and explanation on cybersecurity topics, not for tool-calling agent harnesses.
Compatibility
| Item | Value |
|---|
| Base architecture | NemotronHForCausalLM (Nemotron 3 Super hybrid Mamba–Transformer MoE) |
| Trained against | mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit |
| Format | Apple MLX LoRA (adapters.safetensors) |
| Framework | mlx-lm LoRA tuner |
Load with the matching Nemotron 3 Super MLX checkpoint (prefer a 6-bit MLX build consistent with training). Adapters are not guaranteed to transfer cleanly across different quantization levels of the same base.
Training summary
| Hyperparameter | Value |
|---|
| Training steps (iters) | 1000 (config planned 1500; published checkpoint is iter 1000) |
| Learning rate | 3e-6 |
| Optimizer | Adam |
| Batch size | 1 |
| Max sequence length | 4096 |
| Gradient checkpointing | enabled |
LoRA layers (num_layers) | 16 (upper transformer blocks) |
| LoRA rank | 8 |
| LoRA scale | 20.0 |
| LoRA dropout | 0.0 |
| Checkpoint cadence | every 100 iters |
| Seed | 0 |
| Prompt loss masking | off (mask_prompt: false) |
Hardware context for this run: Apple Silicon unified memory (MLX Metal).
Training data
Supervised fine-tuning on 21,265 prompt / completion pairs from the Mati theory_gemma12b specialist pack (theory_steps.jsonl), built for cybersecurity conceptual coverage (no agent tool traces).
Composition (by source)
| Source | Rows | Role |
|---|
| RedSage-Seed | 14,846 | Offensive/defensive skill writeups, frameworks, Kali tooling, CLI literacy |
| Primus Reasoning (CTIBench / DeepSeek-R1 style) | 2,489 | Longer cybersecurity reasoning traces |
| CTFtime writeups | 2,345 | Competition writeup → teaching explanations |
| CISA KEV | 800 | Known Exploited Vulnerabilities catalog Q&A |
| Primus Instruct | 785 | Security ops tasks (alerts, cmd analysis, docs QA, Terraform misconfig, SIEM-style queries) |
| Total train | 21,265 | |
RedSage subset breakdown (within the 14,846): general 6,924; framework 3,715; skills 2,490; Kali 1,023; CLI 694.
Validation during training used a 389-row valid.jsonl held out for mlx-lm eval (val_batches: 25, steps_per_eval: 200).
Pack construction notes (from the theory build manifest): max output length capped near 6,000 characters per completion during curation; CTFtime and KEV intake capped (2,500 / 800) to keep the mix balanced.
Intended use
- Cybersecurity explanation and mentoring
- Attack-path / CTF concept walkthroughs
- Vulnerability and tooling literacy (KEV, common utilities)
- Domain grounding for a larger multi-adapter stack (theory expert before agentic / code-as-action stages)
Out of scope / limitations
- Not trained as a tool-using coding agent (no Cursor / OpenHands trajectories in this adapter).
- Training rows are SFT prompt/completion pairs from a pack originally assembled for Gemma chat formatting; applied here as raw text targets under
mlx-lm.
- Does not authorize illegal activity. Outputs can be inaccurate; verify against authoritative sources before operational use.
- LoRA is a narrow adapter: base model capabilities outside the fine-tune distribution still dominate.
Files
| File | Description |
|---|
adapters.safetensors | Final LoRA weights (iteration 1000) |
adapter_config.json | Training configuration used by mlx-lm |
Load example (MLX)
1from mlx_lm import load, generate
2
3model, tokenizer = load(
4 "mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit", # or equivalent MLX Nemotron 3 Super build
5 adapter_path="True2456/nemotron-3-super-120b-a12b-theory-lora-mlx",
6)
7
8prompt = "Explain Kerberos unconstrained delegation and why it is dangerous in Active Directory."
9text = tokenizer.apply_chat_template(
10 [{"role": "user", "content": prompt}],
11 tokenize=False,
12 add_generation_prompt=True,
13)
14print(generate(model, tokenizer, prompt=text, max_tokens=512))
CLI:
1python -m mlx_lm generate \
2 --model mlx-community/Nemotron-3-Super-120B-A12B-MLX-6bit \
3 --adapter-path True2456/nemotron-3-super-120b-a12b-theory-lora-mlx \
4 --prompt "Summarize how SAM hive secrets are used post-compromise."
License
Base model: NVIDIA Open Model License (see link in metadata). This adapter inherits that license's restrictions for derived works. Training data mixes public cybersecurity corpora (including CTF writeups and CISA KEV); respect upstream licenses and ToS when redistributing datasets separately.
Citation
If you use this adapter, please also cite NVIDIA Nemotron 3 Super and the upstream theory sources (RedSage, Primus, CTFtime, CISA KEV) as applicable.
