This model is the full-precision version of the Bleta-Logjike 27B model, specifically optimized for logical reasoning tasks in the Albanian language. Bleta is an Albanian adaptation based on Google's Gemma 3 architecture, with this version focused on enhancing logical reasoning and problem-solving capabilities for Albanian speakers.
This model was fine-tuned using Generative Rejection Policy Optimization (GRPO), a reinforcement learning technique that trains models to optimize for specific reward functions. GRPO allows the model to learn from feedback on its generated responses, improving reasoning quality over time by:
The training utilized the GSM8K (Grade School Math 8K) dataset, which contains over 8,000 high-quality grade school math problems, requiring step-by-step reasoning to solve. The dataset provides:
This dataset was adapted for Albanian language training to ensure the model can handle mathematical reasoning tasks in Albanian.
1from transformers import AutoModelForCausalLM, AutoTokenizer
2
3model_name = "klei1/bleta-logjike-27b"
4model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto", load_in_8bit=True)
5tokenizer = AutoTokenizer.from_pretrained(model_name)
6
7messages = [
8 {"role": "user", "content": "Si llogaritet sipërfaqja e një trekëndëshi?"}
9]
10
11text = tokenizer.apply_chat_template(messages, add_generation_prompt=True, tokenize=False)
12inputs = tokenizer(text, return_tensors="pt").to(model.device)
13
14outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.95)
15print(tokenizer.decode(outputs[0], skip_special_tokens=True))
This is the full-precision version of the model requiring significant computational resources. For deployment on consumer hardware, consider using the 8-bit quantized GGUF version available at klei1/bleta-logjike-27b-finetune.
