A fine tuned version of Qwen2.5-0.5B-Instruct trained on the tatsu-lab/alpaca instruction dataset using Unsloth. This project was built as a learning exercise to understand the complete LLM fine tuning workflow, from training and LoRA adaptation to model merging, GGUF conversion, local inference with Ollama, and deployment on Hugging Face.
Model Details
Model Name:ciphermosaic/qwen-alpaca-gguf
Base Model: Qwen2.5-0.5B-Instruct
Model Type: Causal Language Model
Primary Purpose: Chat Assistant
Language: English
License: Apache 2.0
Training Dataset
The model was fine tuned using the tatsu-lab/alpaca dataset, which consists of instruction, input, and output pairs designed for instruction following tasks.
The objective was to improve the model's ability to follow natural language instructions and generate coherent conversational responses.
Fine Tuning
The model was fine tuned using Unsloth with LoRA.
Training Configuration
Parameter
Value
Epochs
2
Learning Rate
2e-5
Fine Tuning Method
LoRA
Framework
Unsloth
After training:
The LoRA adapter was merged into the base model.
The merged model was exported.
The model was converted to GGUF format for efficient local inference.
Quantization
The released model uses the following GGUF file:
Qwen2.5-0.5B-Instruct.Q4_K_M.gguf
The Q4_K_M quantization provides a good balance between model size, inference speed, and response quality, making it suitable for running efficiently on consumer hardware.
Local Inference with Ollama
The model can be run locally using Ollama after creating a Modelfile that references:
Qwen2.5-0.5B-Instruct.Q4_K_M.gguf
Then create and run the model:
bash
1ollama create qwen-alpaca -f Modelfile
2ollama run qwen-alpaca
This enables fast local inference without requiring a cloud API.
Deployment
The project has also been deployed on Hugging Face Spaces, allowing users to interact with the model through a web interface in addition to local inference with Ollama.
Intended Uses
This model is suitable for:
General chat
Instruction following
Learning and experimentation
Prompt engineering practice
Local LLM inference
Educational demonstrations of LLM fine tuning
Limitations
Trained for only two epochs.
No quantitative evaluation metrics were collected.
Responses may occasionally contain factual inaccuracies or hallucinations.
Not intended for production or high risk applications.
Performance is limited by the 0.5B parameter size.
Example
Prompt
Explain what a Transformer is in simple terms.
Response
A Transformer is a neural network architecture designed to process sequences efficiently using self attention. Instead of reading text one word at a time, it looks at all relevant words together, allowing it to understand context better and generate more accurate responses.
Learning Objectives
This project was created to gain hands on experience with the complete LLM development pipeline, including:
Instruction fine tuning
LoRA adaptation
Model merging
GGUF conversion
Quantization
Local inference with Ollama
Hugging Face model hosting
Hugging Face Spaces deployment
Local Inference with Ollama
The model was successfully tested locally using Ollama.
Ollama Demo
Acknowledgements
Qwen Team for the Qwen2.5 base model.
Unsloth for efficient LoRA fine tuning.
tatsu-lab for the Alpaca instruction dataset.
Ollama for local LLM inference.
Hugging Face for model hosting and Spaces deployment.
