🔷 Nexura-Gemma-2B

A Supervised Fine-Tuned + DPO-Aligned Gemma-2B Model

1. SFT (Supervised Fine-Tuning) using high-quality instruction datasets
2. DPO (Direct Preference Optimization) for preference alignment

<user>
{instruction}
</user>

<assistant>
{response}

📌 1. Base Model

• Base: google/gemma-2b
• Architecture: Decoder-only transformer LLM
• Tokenizer: Gemma tokenizer (sentencepiece)
• Training Type: QLoRA (SFT) + DPO
• Language: English
• Usage: General-purpose text generation & instruction following

📌 2. Datasets Used

🟦 A. SFT Dataset (Supervised Fine-Tuning)

• tatsu-lab/alpaca (~52k)
• databricks/dolly-15k
• Additional filtered samples:
  • lamini_20k
  • ign_20k
  • ultrachat_20k
    (mostly skipped due to filtering)

SFT Prompt Format

<user>
{instruction}
</user>

<assistant>
{response}

🟩 B. DPO Dataset (Preference Alignment)

• Anthropic HH-RLHF
• Stanford SHP
• UltraFeedback
• JudgeLM

📌 3. Training Details

🟦 SFT (Supervised Fine-Tuning)

• Rank: 8
• Alpha: 16
• Dropout: 0.05
• Precision: bfloat16
• Epochs: 1
• LR: 2e-4
• Gradient Accumulation: 20
• Target Modules:
  • q_proj, k_proj, v_proj, o_proj
  • gate_proj, up_proj, down_proj

🟩 DPO (Direct Preference Optimization)

• Beta: 0.1
• Learning rate: 5e-5
• Grad Accumulation: 8
• Policy model = SFT-trained adapter

📌 4. Inference Instructions

Prompt Template

<user>
{your_message}
</user>

<assistant>

🟦 FastAPI Streaming Server (server.py)

• Local model loading (no HF auto-download)
• SFT-exact prompt builder
• Tag suppression to prevent invalid XML-like output
• Greedy decoding:
  • do_sample=False
  • repetition_penalty=1.3
  • no_repeat_ngram_size=4

Example: Python Local Inference

🟩 Curl API Example

curl -X POST http://localhost:8000/api/chat \
     -H "Content-Type: application/json" \
     -d '{"messages":[{"role":"user","content":"hi"}]}'

📌 5. Intended Use

✔ Recommended Uses

• Chat assistants
• Instruction following
• Educational Q/A
• Coding help
• Summaries
• Reasoning tasks
• Content rewriting

❌ Not Recommended

• Medical, legal, or financial advice
• Real-world decision making
• High-risk or safety-critical systems
• Generating harmful, biased, or toxic content

📌 6. Strengths

• Lightweight (2B parameters)
• Fast inference on consumer GPUs
• Clean behavior after SFT formatting correction
• Strong alignment after DPO training
• Stable responses due to greedy decoding

📌 7. Limitations

• Limited knowledge compared to larger LLMs
• May hallucinate if prompt format is not followed
• Not multilingual
• No factual updates after 2023 (Gemma limitation)

📌 8. Hardware Requirements

• GPU Recommended: 8GB+ VRAM
• Minimum CPU RAM: 6GB
• Quantized 4-bit mode: Runs on mid-range systems
• Ideal: NVIDIA RTX 3060 / 4060+

📌 9. License

• Research use
• Commercial use under conditions
• Attribution to Google

📌 10. Citation

@misc{nexura_gemma2b_2025,
  title={Nexura-Gemma-2B},
  model={Custom fine-tuned Gemma-2B},
  author={Arun Vpp},
  year={2025},
  publisher={Hugging Face}
}

🎯 Final Notes