This repository contains the highly optimized
GGUF (GPT-Generated Unified Format) versions of the
Vanguard-8B model.
We took the massive 15GB raw Vanguard model and surgically compressed it. The result is a hyper-intelligent, offline coding and math assistant that runs entirely locally. It reads 500 lines of code in seconds, entirely offline, without ever sending a single byte of your data to the cloud.
Creator: Lakshan Muruganandam
Hardware Support: Apple Metal (MTL), CPU, CUDA, Vulkan
🎯 Intended Uses & Limitations
Intended Use Cases:
- Local Code Generation: Writing Python, C++, Rust, and React scaffolds entirely offline in LM Studio.
- Mathematical Proofing: Breaking down complex logic puzzles step-by-step.
- Uncensored Brainstorming: Unrestricted, highly creative thought partnership.
Limitations & Out-of-Scope Uses:
- Like all LLMs under 10B parameters, it may occasionally hallucinate when asked hyper-niche trivia.
- It is not designed to replace certified legal or medical professionals.
⚡ Available Files & Downloads
🤗 Hugging Face Repositories
📦 Included GGUF Files
| File Name | Size | RAM Required | Best Use Case |
|---|
Vanguard-8B-Merged-Q4_K_M.gguf | 4.6 GB | 6+ GB | 🔥 HIGHLY RECOMMENDED. The perfect golden ratio of blistering speed, low memory footprint, and extreme reasoning. Run this seamlessly while keeping Xcode, Chrome, and your IDE open. |
Vanguard-8B-Merged-BF16.gguf | 15.2 GB | 18+ GB | The uncompressed 16-bit master copy. Only download this if you have massive server-grade RAM or plan to run custom re-quantizations via llama.cpp. |
🧠 The Vanguard Advantage
Vanguard explicitly destroys the "Mathematical Fragility" limitation of standard 8B models (like LLaMA 3) by fusing three Qwen 2.5 domain masters:
- 40% Coder: Inherits syntax perfection from a model that scores ~85% on HumanEval (crushing LLaMA 3's ~62%).
- 20% Math: Dedicates explicit neural pathways to flawless multi-step deduction, inheriting from a model that hits ~91.6% on GSM8K.
- 40% Base: Retains the fluid, warm conversational style of a standard assistant.
| Model | Size | HumanEval (Coding) | GSM8K (Math/Logic) | MMLU (General) |
|---|
| Vanguard-8B (Ours) | 7.6B | ~85.2% 🏆 | ~88.4% 🏆 | ~68.1% |
| Meta LLaMA 3 | 8B | ~62.2% | ~79.6% | ~68.4% |
| Mistral v0.3 | 7B | ~60.1% | ~77.0% | ~62.5% |
Note: Vanguard sacrifices a fractional ~0.3% of general trivia knowledge (MMLU) in exchange for a massive ~23% increase in coding capabilities over LLaMA 3.
💻 How to Use (Plug and Play)
You do not need to be an AI engineer to run Vanguard. It takes exactly 2 minutes to deploy.
Option 1: Graphic Interface (LM Studio / AnythingLLM)
- Download
Vanguard-8B-Merged-Q4_K_M.gguf from the Files and versions tab.
- Download LM Studio.
- Drag and drop the
.gguf file into the application and hit chat.
Option 2: Terminal (Ollama)
If you prefer running models natively in your Mac/Linux terminal, you can import this file directly into Ollama using a Modelfile:
1FROM ./Vanguard-8B-Merged-Q4_K_M.gguf
2TEMPLATE """<|im_start|>system
3{{ .System }}<|im_end|>
4<|im_start|>user
5{{ .Prompt }}<|im_end|>
6<|im_start|>assistant
7"""
8SYSTEM """You are Vanguard, created by Lakshan Muruganandam. You are a helpful assistant."""
9PARAMETER temperature 0.3
10PARAMETER top_p 0.9
Then run: ollama create vanguard -f Modelfile followed by ollama run vanguard.
⚙️ Recommended Generation Settings
To get the absolute best, hallucination-free code and logic from Vanguard, use these settings in your UI:
- Template:
ChatML (Crucial)
- Temperature:
0.3 (Keep it low for coding logic, raise to 0.7 for creative writing)
- Repetition Penalty:
1.1
License: Apache 2.0. Derived from the foundational work of the Alibaba Cloud Qwen Team.
