LLaMA 3.2–1B DSP (English–Albanian)
A lightweight LLaMA 3.2–1B model fine-tuned to act as a bilingual (English–Albanian)
Digital Signal Processing (DSP) teaching assistant.
This model is part of a university project exploring whether small modern LLMs can handle
technical DSP reasoning when trained on a verified synthetic dataset using QLoRA.
Intended Use
- Coursework-level DSP questions (FFT, aliasing, sampling)
- Short conceptual explanations in English or Albanian
- Mixed-language prompts (EN ↔ SQ)
- As a compact teaching assistant for students
Not intended for high-stakes or professional engineering decisions.
Training Overview
The model was fine-tuned using:
- QLoRA (4-bit quantisation + LoRA adapters)
- A bilingual DSP dataset (EN/SQ)
- Programmatically generated and numerically verified questions
- Teacher–student refinement using larger LLaMA and Qwen models
Training ran within a ~5–6GB VRAM budget on consumer hardware.
📂 Dataset
The training data includes:
- Manually written English and Albanian DSP questions
- A small English–Albanian DSP glossary
- Synthetic tasks for:
- FFT bin indices
- aliasing frequencies
- basic sampling relations
- Verification scripts that recompute all numeric answers
Only samples that passed numeric checks and basic quality filters were included.
Evaluation (Summary)
The model was evaluated on three pipelines:
| Pipeline | Accuracy (approx.) |
|---|
| English | ~80% |
| Albanian | ~70% |
| Mixed EN/SQ | ~68% |
The fine-tuned model clearly outperforms the base LLaMA 3.2–1B on DSP tasks and comes close to mid-sized models in the 3B–7B range.
How to Use
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model_id = "Irfanuruchi/llama_3_2_1B_dsp-llm-bilingual"
4
5tokenizer = AutoTokenizer.from_pretrained(model_id)
6model = AutoModelForCausalLM.from_pretrained(
7 model_id,
8 device_map="auto"
9)
10
11prompt = "A signal is sampled at 8000 Hz. What FFT bin corresponds to 1000 Hz for N=256?"
12
13inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
14outputs = model.generate(**inputs, max_new_tokens=128)
15
16print(tokenizer.decode(outputs[0], skip_special_tokens=True))
License
These fine-tuned weights are based on meta-llama/Llama-3.2-1B and are therefore governed by the LLaMA 3.2 license.
The training code and surrounding project are released separately under the MIT license on GitHub:
Please make sure you comply with both:
The LLaMA 3.2 license for the model weights, and the MIT license for the code.
Acknowledgements
This model was developed as part of the Introduction to Data Science course at
South East European University.
Thanks to Professor Nuhi Besimi for his guidance and feedback throughout the project.
