A PyTorch implementation of RNN-based Neural Machine Translation system for Chinese-to-English translation, featuring LSTM encoder-decoder architecture with attention mechanisms.
Introduction
This repository implements a RNN-based Neural Machine Translation system with the following key components:
Model: Implement a model using LSTM, with both the encoder and decoder consisting of unidirectional layers.
Attention mechanism: Implement the attention mechanism and investigate the impact of different alignment functions—such as dot-product, multiplicative, and additive—on model performance.
Training policy: Compare the effectiveness of Teacher Forcing and Free Running strategies.
Decoding policy: Compare the effectiveness of greedy and beam-search decoding strategies.
Key Features
Encoder: Unidirectional LSTM encoder for source language (Chinese)
Decoder: Unidirectional LSTM decoder with attention mechanism for target language (English)
Attention Types:
Dot-product attention
Multiplicative attention
Additive attention (Bahdanau-style)
Tokenization:
Chinese: Jieba word segmentation
English: SentencePiece subword tokenization
Training Strategies:
Teacher Forcing (configurable ratio)
Free Running
Decoding Strategies:
Greedy decoding
Beam search decoding (configurable beam size)
Data Preparation
The compressed package contains four JSONL files, corresponding respectively to the small training set, large training set, validation set, and test set, with sizes of 100k, 10k, 500, and 200 samples. Each line in a JSONL file contains one parallel sentence pair. The final model performance will be evaluated based on results on the test set.
