You can deploy this model and start earning money today!

GaryNENE/
nemotron-nano-8b-reasoning-lora

Name: nemotron-nano-8b-reasoning-lora
Availability: InStock

2505.00949

2506.13284

Views

No views yet

Nemotron-Nano-8B Reasoning LoRA

LoRA fine-tuned adapter for the NVIDIA Nemotron Model Reasoning Challenge on Kaggle.

🏗️ Architecture

Base Model: nvidia/Llama-3.1-Nemotron-Nano-8B-v1 (8B params, Llama 3.1 architecture)
Fine-tuning: QLoRA (4-bit NF4 quantization + LoRA rank 128)
Target Modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
LoRA Config: r=128, alpha=64, dropout=0.05, RSLoRA enabled

📊 Training Data (~29K samples)

Dataset	Samples	Description
jasonkung98/NVIDIA-Nemotron-Model-Reasoning-Challenge	9,500	Competition training data (bit manipulation, ciphers, number systems, cryptarithms)
nvidia/Nemotron-RL-ReasoningGym-v1	~14,259	104 diverse puzzle/logic task types (cryptarithms, matrix ops, equations, Tower of Hanoi, palindromes, etc.)
nvidia/AceReason-1.1-SFT	5,000	Math reasoning with real chain-of-thought traces from DeepSeek-R1

🔧 Training Hyperparameters

Parameter	Value
Epochs	3
Batch size (effective)	16 (2 × 8 grad accum)
Learning rate	2e-4 (cosine schedule)
Warmup	5% of steps
Max sequence length	4096
Precision	BF16 + 4-bit NF4 quantization
Optimizer	AdamW
Weight decay	0.01

🚀 Quick Start — Training

bash

# Install dependencies
pip install torch transformers trl peft datasets accelerate bitsandbytes trackio huggingface_hub sentencepiece protobuf

# Run training (requires GPU with >= 16GB VRAM)
python train_nemotron_lora.py

Hardware recommendations:

Kaggle T4 (16GB): ✅ Works with QLoRA
A100 (80GB): Fastest, ~2-3 hours for full training
RTX 3090/4090 (24GB): ✅ Works well

🔮 Quick Start — Inference

python

from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig
from peft import PeftModel
import torch

# Load with QLoRA
bnb_config = BitsAndBytesConfig(
    load_in_4bit=True, bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16, bnb_4bit_use_double_quant=True
)
model = AutoModelForCausalLM.from_pretrained(
    "nvidia/Llama-3.1-Nemotron-Nano-8B-v1",
    quantization_config=bnb_config, torch_dtype=torch.bfloat16, device_map="auto"
)
model = PeftModel.from_pretrained(model, "GaryNENE/nemotron-nano-8b-reasoning-lora")
model.eval()

tokenizer = AutoTokenizer.from_pretrained("nvidia/Llama-3.1-Nemotron-Nano-8B-v1")

# Generate with reasoning toggle ON
messages = [
    {"role": "system", "content": "detailed thinking on"},
    {"role": "user", "content": "Your puzzle/reasoning question here..."},
]
input_text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(input_text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=32768, temperature=0.6, top_p=0.95, do_sample=True)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

📋 Competition Task Types

The Nemotron Reasoning Challenge tests:

🔢 Bit manipulation — Binary transformations (XOR, shifts, rotations)
🔐 Cipher/encryption — Secret substitution rules on text
🔢 Number systems — Roman numerals, base conversion
🧩 Cryptarithms — Letter-to-digit puzzles
🧮 Pattern recognition — Discover hidden rules from examples

📚 Key References

📄 License

This LoRA adapter inherits the license of the base model: NVIDIA Open Model License.