Project Overview
This is to develop a neural network-based language model for next token prediction.
File Structure
nlpmid.ipynb
The main Jupyter notebook that implements the following:
Training in Two Languages: English and the second language (as assigned previously).
Tokenizer: Uses a custom or pre-built tokenizer.
Embedding Model: Includes code for using or training an embedding model.
Model Architecture: Implements one of the allowed architectures (NN, RNN, or LSTM).
Checkpointing: Code to save model checkpoints during training.
Training & Validation Loss Graph: Includes code to plot and analyze the training and validation loss.
Data Files:
JSON Files:
alpaca_data_cleaned (1).json
ChineseSimplified (1).json
These files contain the datasets in English and the second language for training.
CSV File:
training_results.csv: Stores the training and validation loss over epochs, used to generate the graph for training performance analysis.
Checkpoints Folder:
/checkpoints/: A folder where model checkpoints are saved during training to enable loading and resuming the model from the last saved state.
Usage Instructions
Open the nlpmid.ipynb file in Jupyter Notebook.
Ensure the required datasets (alpaca_data_cleaned (1).json, ChineseSimplified (1).json) are available in the specified directory.
Run the cells sequentially to load and clean the data.
The notebook checks for GPU availability to optimize performance.
Requirements
Python 3.x
Jupyter Notebook
PyTorch
JSON datasets (alpaca_data_cleaned (1).json, ChineseSimplified (1).json)
