Views
No views yet
README.md for Hugging Face Model Card1
2datasets:
3 - custom
4library_name: onmt
5model-index:
6 - name: Hindi to Malayalam Translation
7 results:
8 - task:
9 name: Translation
10 type: translation
11 dataset:
12 name: Custom Hindi- Malayalam Parallel Corpus
13 type: translation
14 metrics:
15 - name: BLEU
16 type: bleu
17 value: 11.07
18 - name : COMET
19 - type:comet
20 - value: 0.832
21---
22
23Hindi to Malayalam Translation Model (OpenNMT)
24
25This is a Neural Machine Translation (NMT) model trained to translate Hindi (hi) to Malayalam (ml) using the OpenNMT framework. It was trained on a custom curated low-resource parallel corpus.
26
27Model Architecture
28
29- Framework: OpenNMT (PyTorch)
30- Architecture: Transformer
31- Type: Sequence-to-sequence
32- Layers: 6 encoder / 6 decoder
33- Embedding size: 512
34- FFN size: 2048
35- Attention heads: 8
36- Positional encoding: sinusoidal
37- Tokenizer: SentencePiece (trained jointly on hi-ml)
38- Vocabulary size: 32,000 (joint BPE)
39
40Training Details
41
42| Setting | Value |
43|----------------------|------------------------|
44| Framework | OpenNMT (3.5.1) |
45| Training steps | 800k |
46| Optimizer | Adam + inverse sqrt LR |
47| Batch size | 8192 tokens |
48| Max tokens | 8192 |
49| Dropout | 0.3 |
50| BLEU (test set) | 11.07 |
51| Hardware | 1 x V100 32GB GPU |
52| Training time | ~15 hours |
53
54Evaluation
55
56The model was evaluated on a manually annotated Hindi-Malayalam test set consisting of 10,000 sentence pairs.
57
58| Metric | Score |
59|--------|---------|
60| BLEU | 11.07 |
61| COMET | 0.832 |
62
63Usage
64
65IN CLI
66
67```bash
68onmt_translate \
69 -model hi_ml_onmt.pt \
70 -src input.txt \
71 -output output.txt \
72 -replace_unk \
73 -verbose \
74 -gpu -1 \
75 -min_length 11import torch
2import onmt.model_builder
3import onmt.inputters
4import onmt.opts
5
6# Path to your model
7model_path = "model.tm_best_checkpoint.pt"
8
9# Load the checkpoint (map to CPU or GPU based on availability)
10checkpoint = torch.load(model_path, map_location=torch.device("cpu"))
11
12# Extract model options and vocab/fields
13model_opt = checkpoint['opt']
14fields = checkpoint['vocab']
15
16# Build the model using OpenNMT-py utilities
17model = onmt.model_builder.build_base_model(model_opt, fields, use_gpu=False, checkpoint=checkpoint)
18
19# Set model to eval mode
20model.eval()
21