You can deploy this model and start earning money today!

phonsobon/
khmer-images_captioning

Name: khmer-images_captioning
Availability: InStock

phonsobon/khmer_images_captioning_v2

mit

Views

No views yet

Khmer Image Captioning (Show, Attend and Tell)

A from-scratch image captioning model for Khmer, following the classic Show, Attend and Tell architecture:

Encoder: ResNet-101 pretrained on ImageNet, used only as a frozen feature extractor (no fine-tuning) producing a 14x14 grid of 2048-dim region features.
Decoder: an LSTM with Bahdanau (additive) attention, embedding table, and gating layers, trained from scratch on Khmer captions.
Tokenization: word-level, using khmer-nltk for Khmer word segmentation.

Trained on phonsobon/khmer_images_captioning_v2.

This is not a standard transformers model — it uses custom PyTorch code (modeling_khmer_captioning.py in this repo) rather than AutoModel.

Files

config.json — architecture hyperparameters
decoder.pt — trained decoder weights (state_dict)
vocab.json — word-level vocabulary (itos list)
modeling_khmer_captioning.py — model classes + load_model() / caption_image() helpers

The encoder is not included — it's just off-the-shelf frozen ImageNet ResNet-101 weights, downloaded automatically via torchvision when you load the model.

Usage

pip install torch torchvision pillow huggingface_hub

python

from huggingface_hub import snapshot_download
import sys

repo_dir = snapshot_download("phonsobon/khmer-images_captioning")
sys.path.insert(0, repo_dir)

from modeling_khmer_captioning import load_model, caption_image

encoder, decoder, itos, stoi, cfg = load_model(repo_dir)
caption = caption_image("your_image.jpg", encoder, decoder, itos, stoi, cfg, beam_size=3)
print(caption)

Limitations

Trained on a relatively small dataset (~3.7k image-caption pairs), so generalization outside that domain will be limited.
Khmer text has no spaces between words; captions are segmented with khmer-nltk, whose probabilistic tokenizer won't always agree exactly with human segmentation, which affects BLEU-style evaluation more than it affects readability.