It is trained using LoRA (Low-Rank Adaptation) on a subset of the COCO Thai caption dataset saksornr/coco_caption-thai-ipu24-train-sample10k to generate Thai caption from images.

Model Description

Base Model: Qwen3-VL-8B-Instruct (4-bit quantized via Unsloth)
Fine-tuning Method: LoRA (PEFT)
Task: Image Captioning (Thai)
Language: Thai
Framework: Transformers + TRL + Unsloth

Training Details

Dataset

Name: saksornr/coco_caption-thai-ipu24-train-sample10k
A Thai-translated version of COCO captions
~10,000 samples used for training

Training Setup

Method: Supervised Fine-Tuning (SFT)
Adapter: LoRA
Quantization: 4-bit (bnb)
Frameworks: TRL + Transformers + Unsloth
Training code is released here

Usage

from PIL import Image
from unsloth import FastVisionModel
model, tokenizer = FastVisionModel.from_pretrained(
    "unsloth/Qwen3-VL-8B-Instruct-unsloth-bnb-4bit",
    load_in_4bit = True,
)
model.load_adapter("KiruruP/Qwen3-VL-8B-ipu10k-thaicaption-lora")
FastVisionModel.for_inference(model)

instruction = "จงบรรยายภาพนี้เป็นภาษาไทยแบบสั้น กระชับ และเป็นธรรมชาติ"
messages = [
    {"role": "user", 
    "content": [
        {"type": "image"},
        {"type": "text", "text": instruction}
    ]}
]
input_text = tokenizer.apply_chat_template(messages, add_generation_prompt = True, tokenize = False)

image = Image.open("example.jpg").convert("RGB")
inputs = tokenizer(
    image,
    input_text,
    add_special_tokens = False,
    return_tensors = "pt",
).to("cuda")
output = model.generate(**inputs, max_new_tokens = 100)
generated_tokens = outputs[0][inputs.input_ids.shape[1]:] # For filtering prompt in output
caption = tokenizer.decode(generated_tokens, skip_special_tokens=True).strip()
print(caption)