This repository contains an optimized 4-bit GGUF (q4_k_m) instance of a fine-tuned Qwen2.5-3B-Instruct model, specialized for autonomous telecom workflows and precise eSIM package recommendations.

The model was trained via Unsloth using QLoRA parameters and compiled natively through llama.cpp hooks to enable ultra-low latency inference on local commodity CPUs and budget cloud servers.

🎯 Model Capabilities

eSIM Plan Recommendation: Accurately matches user destinations and travel durations with precise package specifications.
Dynamic Link Preservation: Natively handles and injects structural production checkout links (https://esim.oss.net.bd/checkout/...) without formatting degradation.
Strict ChatML Adherence: Fine-tuned to enforce clear boundaries for multi-turn conversational agents.

🚀 Quantization Metrics

Format: GGUF (q4_k_m - Recommended balanced medium quantization)
VRAM/RAM Footprint: ~1.8GB to 2GB (Ideal for low-tier VPS environments, local setups, or mobile edge testing)
Quality Loss: Minimal parameter disruption compared to full-precision weights.

🔧 Local Deployment via Ollama

To execute this compiled model locally or interface it with your backend FastAPI microservices using Ollama, implement the following setup:

Download the .gguf file from the Files section.
In the same directory, create a configuration file named Modelfile and input the following block:

text

FROM ./qwen2.5-3b-esim-assistant-Q4_K_M.gguf

# Define ChatML Template Boundaries
TEMPLATE """{{ if .System }}<|im_start|>system
{{ .System }}<|im_end|>
{{ end }}{{ if .User }}<|im_start|>user
{{ .User }}<|im_end|>
{{ end }}<|im_start|>assistant
{{ .Output }}<|im_end|>"""

PARAMETER stop "<|im_end|>"
PARAMETER stop "<|im_start|>"

sourav-paramanya/qwen2.5-3b-esim-assistant-GGUF

Qwen2.5-3B-eSIM-Assistant-GGUF

🎯 Model Capabilities

🚀 Quantization Metrics

🔧 Local Deployment via Ollama

sourav-paramanya/
qwen2.5-3b-esim-assistant-GGUF