English-Czech Machine Translation Project

Overview

Technical Implementation

Data Processing

• Format: Parallel English-Czech text corpus
• Processing Steps:
  • Cleaning and normalization
  • Tokenization using HuggingFace tokenizers
  • Train-validation split
  • Conversion to PyTorch tensors

Model Architectures

1. LSTM Model

• Simple LSTM architecture for baseline performance
• Parameters:
  • Hidden size: 128
  • Batch size: 32
  • Learning rate: 0.0001
  • Optimizer: Adam
  • Loss function: CrossEntropyLoss

2. Seq2Seq Model (No Attention)

• Encoder-Decoder architecture
• Optimized Implementation:
  • Hidden size: 256
  • Batch size: 512
  • Optimizer: AdamW with OneCycleLR scheduler
  • Mixed precision training
  • Gradient accumulation steps: 4

Performance Optimization

• GPU memory management
• Mixed precision training
• Efficient data loading
• Gradient accumulation
• CUDA optimizations enabled

Evaluation Metrics

• BLEU Score calculation
• CHRF Score measurement
• Training and Validation losses
• Per-epoch metrics recording

Project Structure

/content/drive/MyDrive/
├── Processed Dataset Files
│   ├── train_dataset.pt
│   └── val_dataset.pt
├── LSTM Results
│   ├── Model checkpoints
│   ├── Metrics CSV
│   └── Loss plots
└── Seq2Seq Results
    ├── Model checkpoints
    ├── Metrics CSV
    └── Training curves

Performance Tracking

• Per-epoch model checkpoints
• Comprehensive metrics logging
• Training curve visualization
• Translation quality assessment

Dependencies

• PyTorch
• Transformers
• SacreBLEU
• Pandas
• Matplotlib