EduMentor Qwen3 4B v2 (GGUF)
This repository contains GGUF quantized versions of EduMentor Qwen3 4B v2, an AI engineering mentor designed for university students, technical learning, coding assistance, project guidance, and placement preparation.
These models are converted from the merged FP16 checkpoint and optimized for inference with llama.cpp, LM Studio, Ollama (GGUF import), Open WebUI, Jan, and other GGUF-compatible runtimes.
Base Model
FP16 Repository
Available Quantizations
| File | Size | Recommended For |
|---|
| EduMentor-Qwen3-4B-v2-F16.gguf | ~7.5 GB | Maximum quality |
| EduMentor-Qwen3-4B-v2-Q8_0.gguf | ~4.0 GB | Near-FP16 quality |
| EduMentor-Qwen3-4B-v2-Q6_K.gguf | ~3.1 GB | Best balance of quality and speed |
| EduMentor-Qwen3-4B-v2-Q4_K_M.gguf | ~2.4 GB | Fast local inference |
Features
EduMentor v2 is fine-tuned for engineering education and technical mentoring.
It supports:
- Programming Fundamentals
- Data Structures & Algorithms
- System Design
- Machine Learning
- Deep Learning
- Large Language Models
- Electronics
- Electrical Engineering
- Mechanical Engineering
- Civil Engineering
- Mathematics
- Resume Reviews
- Internship Preparation
- Placement Preparation
- Interview Guidance
- Project Mentoring
Response Format
EduMentor is optimized for multimodal tutoring systems.
Typical responses follow the structure:
1{
2 "speech": "...",
3 "display": {
4 "type": "code | notes | roadmap | table | diagram",
5 "content": "..."
6 },
7 "follow_up": "..."
8}
This enables voice assistants to speak only natural explanations while rendering code, diagrams, or notes separately.
Example
User
Explain Binary Search.
Assistant
1{
2 "speech": "Binary Search repeatedly divides the search interval in half, making it much faster than linear search on sorted arrays.",
3
4 "display": {
5 "type": "code",
6 "language": "python",
7 "content": "def binary_search(...): ..."
8 },
9
10 "follow_up": "Would you like to understand its time complexity?"
11}
Running with llama.cpp
1./llama-cli \
2-m EduMentor-Qwen3-4B-v2-Q4_K_M.gguf \
3-c 4096 \
4-p "Explain recursion simply."
Recommended Quantization
| Hardware | Recommendation |
|---|
| 8 GB RAM | Q4_K_M |
| 16 GB RAM | Q6_K |
| 24 GB+ RAM | Q8_0 |
| GPU Servers | F16 |
Limitations
EduMentor may occasionally:
- produce incorrect technical information
- require verification for safety-critical engineering tasks
- generate imperfect JSON formatting for complex prompts
- benefit from retrieval augmentation for rapidly changing topics
The model should not replace professional engineering advice.
Training
Base Architecture:
Fine-tuning:
- Supervised Fine-Tuning (LoRA)
- Fully merged into FP16
- Converted to GGUF using llama.cpp
Creator
Praneet N S
EduMentor is an ongoing effort to build an AI mentor capable of providing high-quality engineering education through natural conversations and voice-first tutoring systems.
