This model is a fine-tuned variant of mistralai/Mistral-7B-Instruct-v0.3, trained on PalGeo a specialized dataset that focuses on Palestine’s geography.

Developed by: Ahmad Budairi, Belal Hamdeh, Mitri Khoury
Language(s) (NLP): Arabic
Finetuned from model: mistralai/Mistral-7B-Instruct-v0.3

Contact

If you have any feedback, questions, comments, or constructive criticism about this model. please feel free to reach out via the community tab or by email:

✉️ Email: ahmadbudairi333@gmail.com

Training Hyperparameters

Parameter	Value
Lora_r	32
Lora_dropout	0.1
Bnb_4bit_quant_type	“nf4”
num_train_epoch	3
per_device_train_batch_size	4
per_device_eval_batch_size	4
gradient_checkpointing	True
learning_rate	4e-4
optim	“paged_adamw_8bit”
lora_alpha	64
bnb_4bit_compute_dtype	“bfloat16”
bf16	True
gradient_accumulation_steps	64
weight_decay	0.001

Technical Specifications

Hardware

A single NVIDIA GeForce RTX 4070

abulbudz/mistral-palestinian-assistant

Model Description

Contact

Training Hyperparameters

Technical Specifications

Hardware

abulbudz/
mistral-palestinian-assistant