You can deploy this model and start earning money today!

GzimJusufi/
macedonian-qwen2.5-7b

Name: macedonian-qwen2.5-7b
Availability: InStock

safetensors

Views

No views yet

Macedonian Qwen2.5-7B

A Qwen2.5-7B model fine-tuned on Macedonian text — one of the most underserved languages in NLP.

Model Details

Base model: Qwen/Qwen2.5-7B
Fine-tuning method: QLoRA (4-bit quantization)
Training data: 10,000 Macedonian documents (Wikipedia, mC4, Helsinki-NLP)
Training time: ~3.5 hours on Tesla T4

Results

Model	Perplexity
Qwen2.5-7B (base)	10.64
This model	5.21
Improvement	51%

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-7B")
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-7B",
    torch_dtype=torch.float16,
    device_map="auto"
)
model = PeftModel.from_pretrained(model, "GzimJusufi/macedonian-qwen2.5-7b")

inputs = tokenizer("Македонија е земја во", return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Training Details

LoRA rank: 16
LoRA alpha: 32
Optimizer: adamw_8bit
Learning rate: 2e-4
Epochs: 1

GzimJusufi/macedonian-qwen2.5-7b