You can deploy this model and start earning money today!

emre/
gemma-7b-it-Turkish-Reasoning-FT-smol

Name: gemma-7b-it-Turkish-Reasoning-FT-smol
Availability: InStock

emre/finance-reasoning-turkish

unsloth/gemma-7b-it-bnb-4bit

finetune

afl-3.0

Views

No views yet

Model Card for emre/gemma-7b-it-Turkish-Reasoning-FT-smol

Model Details

Model Name: emre/gemma-7b-it-Turkish-Reasoning-FT-smol
Developed by: Davut Emre Tasar
Base Model: unsloth/gemma-7b-it-bnb-4bit
License: Apache-2.0 (Note: The config mentions AFL-3.0, but the uploaded model specifies Apache-2.0; assuming Apache-2.0 as per the uploaded model card.)
Languages:
- Turkish (tr): Primary language for input and output.
- English (en): Used in dataset reasoning components.
Tags: text-generation-inference, transformers, unsloth, gemma, trl
Datasets: emre/finance-reasoning-turkish (Hugging Face dataset)
Fine-Tuning Method: Supervised Fine-Tuning (SFT) with LoRA (Low-Rank Adaptation)
Training Framework: Hugging Face TRL library with Unsloth

This model is a fine-tuned version of the instruction-tuned unsloth/gemma-7b-it-bnb-4bit model, adapted for advanced reasoning tasks in Turkish, focusing on finance-related question-answering. It leverages the emre/finance-reasoning-turkish dataset and was trained using efficient techniques to enhance performance on a single A100 GPU.

Model Usage

This model is designed for text generation, particularly for answering Turkish-language questions with reasoning components. It can be loaded and used with the Hugging Face transformers library. Below are instructions for loading the model and generating responses (code snippets provided separately).

Prerequisites

Python 3.11+
transformers library (version 4.49.0 recommended)
torch library with CUDA support for GPU usage
unsloth library (optional, for faster inference if compatible)

Loading the Model and Tokenizer

You can load the model and tokenizer from the Hugging Face Hub. The model uses 4-bit quantization, so ensure your environment supports bitsandbytes.

Generating Responses

The model expects input in a conversational format, typically with a user question and optional context (e.g., RAG content). It generates responses with reasoning steps, suitable for finance-related queries in Turkish.

Training Procedure

Dataset

Name: emre/finance-reasoning-turkish
Description: A custom dataset containing Turkish finance-related questions, answers, and reasoning components, likely generated or curated for advanced reasoning tasks.

Training Configuration

Optimizer: adamw_8bit
Learning Rate: 2e-4 (initial), with linear scheduling
Batch Size: Effective batch size of 8 (per_device_train_batch_size=2, gradient_accumulation_steps=4)
Epochs: 2
Warmup Steps: 5
Weight Decay: 0.01
Mixed Precision: FP16 (fp16=True)
LoRA Configuration:
- Rank (r): 16
- Alpha (lora_alpha): 16
- Dropout (lora_dropout): 0
- Target Modules: ["q_proj", "o_proj", "gate_proj", "v_proj", "up_proj", "down_proj", "k_proj"]
- Task Type: CAUSAL_LM
Hardware: Single A100 GPU (40GB)

Training Metrics

Total Steps: 464
Training Loss: 0.7098 (final), 0.8503 (average)
Gradient Norm: 0.7641
Runtime: ~19,928 seconds (~5.5 hours)
Samples per Second: 0.186
Steps per Second: 0.023
Total FLOPs: 267,448,136,380,698,620

The model was trained using the Hugging Face TRL library with Unsloth optimizations, achieving 2x faster training compared to standard methods.

Evaluation

Quantitative evaluation metrics (e.g., validation loss) are not explicitly provided in the config, but the training loss decreased to 0.7098 by the end of training, indicating convergence. For reasoning tasks, qualitative evaluation is recommended:

Assess the coherence and relevance of generated responses.
Check for correct reasoning steps in Turkish finance-related outputs.

Ethical Considerations

Bias: The model may inherit biases from the base model or dataset, particularly in financial contexts. Users should review outputs for fairness.
Intended Use: Designed for academic and research purposes. Not suitable for commercial financial advice due to licensing and potential limitations.
Limitations: High computational requirements (GPU recommended) and potential overfitting to the finance domain.

Additional Information

Repository: https://huggingface.co/emre/gemma-7b-it-Turkish-Reasoning-FT-smol
Dataset: https://huggingface.co/datasets/emre/finance-reasoning-turkish
Contact: [huggingface.co/emre]

Useage

1. Loading the Model and Tokenizer

python

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load the tokenizer
tokenizer = AutoTokenizer.from_pretrained("emre/gemma-7b-it-Turkish-Reasoning-FT-smol")

# Load the model with 4-bit quantization
model = AutoModelForCausalLM.from_pretrained(
    "emre/gemma-7b-it-Turkish-Reasoning-FT-smol",
    torch_dtype=torch.float16,
    load_in_4bit=True
)

# Move to GPU if available
device = "cuda" if torch.cuda.is_available() else "cpu"
model.to(device)

2. Generating Responses

python

# Define the input text (example in Turkish)
input_text = (
    "<user_question>Bankalar neden yüksek faiz oranları sunuyor?</user_question>\n"
    "<RAG Content>Bankalar, mevduat toplamak ve krediler için fon sağlamak amacıyla faiz oranlarını "
    "artırabilir. Yüksek faiz oranları, genellikle ekonomik belirsizlik veya enflasyon dönemlerinde "
    "görülür. Bu, yatırımcıları çekmek ve likiditeyi artırmak için bir stratejidir.</RAG Content>\n"
)

# Tokenize the input
inputs = tokenizer(input_text, return_tensors="pt").to(device)

# Generate response
with torch.no_grad():
    outputs = model.generate(
        **inputs,
        max_length=500,  # Adjust as needed
        temperature=0.7,
        top_p=0.9,
        do_sample=True,
        num_return_sequences=1
    )

# Decode and print the response
response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

Citation

If you use this model, please cite it as:

bibtex

@misc{emre_gemma_7b_it_turkish_reasoning_ft_smol,
  author = {Davut Emre Tasar},
  title = {Gemma-7B-IT Turkish Reasoning FT Smol},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/emre/gemma-7b-it-Turkish-Reasoning-FT-smol}}
}