This model is a fine-tuned version of Qwen/Qwen3.5-0.8B. It was trained with LoRA using Unsloth and TRL for English → Kannada translation on the pavan-naik/mt-nllb-en-kn dataset.

Quick start

python

import torch
from unsloth import FastLanguageModel

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="kshitijdesai99/Qwen-3.5-finetuned_mt-nllb-en-kn",
    max_seq_length=512,
    dtype=torch.bfloat16,
    load_in_4bit=False,
)
FastLanguageModel.for_inference(model)

messages = [
    {
        "role": "system",
        "content": "You are a helpful assistant. Translate the given English text to Kannada. Output only the Kannada translation, nothing else.",
    },
    {"role": "user", "content": "Translate to Kannada: Please intercede for us all and especially at the hour of our death, Amen."},
]

prompt = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True,
    enable_thinking=False,
)

inputs = tokenizer(text=[prompt], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=128, do_sample=False)
decoded = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(decoded.strip())

Model Details

Base model: Qwen/Qwen3.5-0.8B
Task: English → Kannada translation
Method: LoRA (not QLoRA)
Quantization: none
Precision: bfloat16
Hardware: NVIDIA H100 NVL
Dataset: pavan-naik/mt-nllb-en-kn
Splits: 16,702 train / 4,017 validation

Training procedure

This model was trained with supervised fine-tuning (SFT) using LoRA adapters on top of the full-precision Qwen/Qwen3.5-0.8B base model.

Training hyperparameters

LoRA rank: 16
LoRA alpha: 16
LoRA dropout: 0.0
Train batch size: 64
Eval batch size: 64
Gradient accumulation: 2
Effective batch size: 128
Epochs: 3
Learning rate: 2e-4
Warmup ratio: 0.05
Logging / eval / save steps: 10 / 100 / 100

Training results

Final train loss: 0.7813
Final eval loss: 0.8166
Training runtime: 1172.14s (~19m 32s)
Train samples/sec: 42.747
Train steps/sec: 0.335

Evaluation

Validation set evaluation was run on all 4,017 examples.

Model	chrF2
Zero-shot `Qwen/Qwen3.5-0.8B`	`20.57`
This LoRA fine-tuned model	`48.15`

Absolute improvement over zero-shot: +27.58 chrF2

Notes

The run used xformers fallback; Flash Attention 2 was not active.
The adapter was trained with standard LoRA, not QLoRA.
The best recorded eval loss was at the final checkpoint of the 3-epoch run.

Framework versions

PEFT 0.19.0
TRL: 0.24.0
Transformers: 5.5.0
Pytorch: 2.10.0
Datasets: 4.3.0
Tokenizers: 0.22.2

Citations

If you use this adapter, please also cite this project repository:

bibtex

@software{desai2026finetuneme,
  author = {Kshitij Desai},
  title = {finetune-me},
  year = {2026},
  url = {https://github.com/kshitijdesai99/finetune-me.git}
}

kshitijdesai99/Qwen-3.5-0.8B-finetuned_mt-nllb-en-kn

Model Card for Qwen-3.5-0.8B-finetuned_mt-nllb-en-kn