his model is based on the ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1 base model and has been fine-tuned on a Turkish Q&A dataset concerning university regulations. The fine-tuning was performed using the LoRA method with 4-bit quantization via BitsAndBytes to reduce memory footprint while preserving performance. The training data, sourced from the dataset yusufbaykaloglu/University_Mevzuat_QA_v2, contains question–answer pairs formatted as follows:

<s>[INST] {question} [/INST] {answer}</s>

Developed by: [Yusuf Baykaloğlu]
Model type: [Causal Language Model fine-tuned with LoRA]
Language(s) (NLP): [Turkish]
Finetuned from model: [ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1]

How to Get Started with the Model

Below is a sample code snippet to get started with the model:

python

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

model_name = "yusufbaykaloglu/Turkish-University-Regulations"

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name)
model.to(device)

input_text = "<s>[INST] Sorunuzu giriniz. [/INST]<s>"
inputs = tokenizer(input_text, return_tensors="pt").to(device)
outputs = model.generate(inputs["input_ids"], max_length=256)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Example

python

input_text = "<s>[INST] Dokuz Eylül Üniversitesi Fen Fakültesi öğrencisiyim. İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için hangi şartları sağlamam gerekiyor?  [/INST]<s>"

<s>[INST] Dokuz Eylül Üniversitesi Fen Fakültesi öğrencisiyim. İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için hangi şartları sağlamam gerekiyor?

[/INST]<s>Madde 6'ya göre, İngilizce hazırlık programından muaf olmak veya doğrudan atlamak için aşağıdaki şartları sağlamanız gerekiyor:
  1. **Yabancı Dil Yeterlik Sınavı:** Yabancı dil yeterliliğinizi kanıtlamak için Yabancı Dil Yeterlik Sınavı'na girmelisiniz.
  2. **Yeterlik Notu:** Sınavda belirli bir not almanız gerekiyor. Bu not, İngilizce hazırlık programından muafiyet veya atlamayı sağlayacak düzeyde belirlenmiştir.
  3. **Muafiyet veya Atlamayı Başvurma:** Sınav sonuçlarınızı ve gerekli belgeleri, fakültenin belirlediği prosedürlere uygun olarak Öğrenci İşleri Daire Başkanlığına sunmanız gerekir.
  4. **Onay:** Öğrenci İşleri Daire Başkanlığı, başvurunuzu inceleyecek ve muafiyet veya atlamayı onaylayacaktır. 

Bias, Risks, and Limitations

This model has been trained on a specific Q&A dataset related to Turkish university regulations and may reflect inherent biases or gaps present in the source data. Users should be aware that:

Generated responses might occasionally be incorrect, incomplete, or outdated.
The model is best used as an assistive tool rather than a sole authority on university regulations.

Training Details

Training Data

Source: The model was fine-tuned using the dataset yusufbaykaloglu/University_Mevzuat_QA_v2, which comprises Turkish question–answer pairs on university regulations.

Training Procedure

Base Model:
- ytu-ce-cosmos/Turkish-Llama-8b-DPO-v0.1
Fine-Tuning Method:
- LoRA (Low-Rank Adaptation)
LoRA Configuration:
- r = 64
- lora_alpha = 128
- lora_dropout = 0.1
- bias = "none"
- Task type: CAUSAL_LM
Training Framework:
- SFTTrainer from the TRL library
Training Hyperparameters:
- Epochs: 8
- Per-device Batch Size: 2
- Gradient Accumulation Steps: 16
- Learning Rate: 1e-4
- Weight Decay: 0.001
- Warmup Ratio: 0.03 (with 100 warmup steps)
- LR Scheduler: Cosine
- Max Sequence Length: 256 tokens
- Optimizer: paged_adamw_32bit
- Max Gradient Norm: 0.3
Quantization:
- 4-bit quantization using BitsAndBytes for efficient memory and compute usage

Training Loss

Below are the training loss values recorded at various training steps:

Step	Training Loss
500	1.077800
1000	0.866600
1500	0.791500
2000	0.736100
2500	0.693900
3000	0.658500
3500	0.642400

Contributions and Support

For any inquiries, bug reports, or collaboration requests, please contact:

Email: Email
Hugging Face Profile: https://huggingface.co/yusufbaykaloglu
GitHub: yusufbaykal
Linkedin:yusufbaykaloglu

Citation

If you use this model in your research or project, please cite it as follows:

bibtex

@misc{Baykaloglu2025TurkishUniversityRegulations,
  author = {Yusuf Baykaloglu},
  title = {Turkish-University-Regulations: A Fine-tuned Turkish Llama Model for University Regulations Q&A},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/yusufbaykaloglu/Turkish-University-Regulations}}
}