This repository contains a fine-tuned Flan-T5 model on the Alpaca Dhivehi dataset, aimed at enabling Dhivehi language instruction-following tasks.

Note: The model can follow instructions and inputs to some extent, but it’s not strictly trained for perfect adherence. Outputs may be partially aligned but are not guaranteed to be fully accurate. Treat results as experimental.

Model Details

Base model: google/flan-t5-small (or whichever size you used)
Dataset: Alpaca Dhivehi , Translation from English to Dhivehi
Training epochs: 5
Final evaluation:
- eval_loss: 2.59
- ROUGE-1: 0.10
- ROUGE-2: 0.03
- ROUGE-L: 0.107

Usage

To run inference using the fine-tuned model:

python

import torch
from transformers import T5Tokenizer, T5ForConditionalGeneration

MODEL_PATH = "alakxender/flan-t5-base-alpaca-dv5"

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

tokenizer = T5Tokenizer.from_pretrained(MODEL_PATH)
model = T5ForConditionalGeneration.from_pretrained(MODEL_PATH).to(device)

def generate_response(instruction, input_text):
    combined_input = f"{instruction.strip()} {input_text.strip()}" if input_text else instruction.strip()
    inputs = tokenizer(combined_input, return_tensors="pt", truncation=True, max_length=256).to(device)
    
    output_ids = model.generate(
        **inputs,
        max_new_tokens=256,
        num_beams=8,
        repetition_penalty=1.5,
        no_repeat_ngram_size=3,
        do_sample=True,
        early_stopping=True,
        temperature=0.1
    )
    
    decoded_output = tokenizer.decode(output_ids[0], skip_special_tokens=True)
    return decoded_output

# Example usage:
instruction = "ދީފައިވާ މައުޟޫޢާ ބެހޭގޮތުން ކުރު ޕެރެގްރާފެއް ލިޔެލާށެވެ."
input_text = "އިއާދަކުރަނިވި ހަކަތަ ބޭނުންކުރުމުގެ މުހިންމުކަން"
print(generate_response(instruction, input_text))

އިއާދަކުރަނިވި ހަކަތަ ބޭނުންކުރުމުގެ މުހިންމު އެއް މައުޟޫއަކީ ސޯލާ، ވިންޑް، ހައިޑްރޯ، ޖިއޮތަރމަލް، އަދި ހައިޑްރޯއިލެކްޓްރިކް ޕަވަރ ފަދަ އިއާދަކުރަނިވި ހަކަތައިން ގްރީންހައުސް ގޭސްތައް ބޭރުވުން .....

Evaluation Results

From the last evaluation:

{
'eval_loss': 1.947984218597412,
'eval_rouge1': 0.19402897739527525,
'eval_rouge2': 0.13442865174751445,
'eval_rougeL': 0.19137712018920253,
'eval_rougeLsum': 0.192215538761906,
'eval_runtime': 46947.7862,
'eval_samples_per_second': 0.11,
'eval_steps_per_second': 0.11,
'epoch': 10.0
}

Notes

This fine-tuned model is experimental and intended for research on Dhivehi-language instruction-following tasks.

alakxender/flan-t5-base-alpaca-dv-ext

Alpaca Dhivehi Fine-Tuned Flan-T5 with an Extended Tokenizer

Model Details

Usage

Evaluation Results

Notes

alakxender/
flan-t5-base-alpaca-dv-ext