This repository contains a fine-tuned LoRA adapter for evaluating TOEFL speaking responses. The model has been trained to assess speaking quality and provide scores based on TOEFL evaluation criteria.

Base Model: Llama 3.2 Training Method: LoRA (Low-Rank Adaptation) Task: TOEFL Speaking Assessment

Repository Contents

Model Weights

toefl_judge_adapter/ - LoRA adapter weights in safetensors format
- Multiple checkpoint files (every 100 steps from 100-1000)
- Final adapter: adapters.safetensors
- Configuration: adapter_config.json

Training Data

train.jsonl (1.9MB) - Training dataset
valid.jsonl (642KB) - Validation dataset
toefl_speaking_data_test.jsonl (650KB) - Test dataset

Application Code

toefl_judge_app.py - Web application for TOEFL evaluation
cli.py - Command-line interface
fine_tune.py - Fine-tuning script
data_formatter.py - Data preprocessing utilities
evaluator.py - Model evaluation tools

Results & Evaluation

toefl_predictions.csv - Model predictions on test set
toefl_evaluations.csv - Evaluation metrics
toefl_evaluation_results.png - Visualization of results

Configuration

lora_config.json - LoRA training configuration

Requirements

Platform: Apple Silicon Mac (M1/M2/M3) - This application uses MLX framework Python: 3.9 or higher

Dependencies

Install the required packages:

pip install -r requirements.txt

Or manually install:

pip install streamlit mlx mlx-lm pandas transformers peft torch

Installation & Setup

Clone or download this repository
Install dependencies:

pip install -r requirements.txt
Download the base Llama 3.2 model:

The application requires the base Llama 3.2 model. You can download it from:
- Hugging Face: meta-llama/Llama-3.2-3B or meta-llama/Llama-3.2-1B
- You'll need to request access from Meta if you haven't already
Update model paths in the application code:

Edit toefl_judge_app.py and set the correct paths for:
- model_path: Path to your base Llama 3.2 model
- adapter_path: Path to the downloaded adapter (from this repo)

Usage

Loading the Model

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Load base model
base_model = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-3.2-3B")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-3B")

# Load LoRA adapter
model = PeftModel.from_pretrained(base_model, "Ylemnox/llama-toefl-checkbot/toefl_judge_adapter")

Running the Web Application

streamlit run toefl_judge_app.py

The web interface will open in your browser where you can:

Enter TOEFL speaking questions
Input student responses
Get automated evaluations with scores (0-4 scale)

Using the CLI

python cli.py

Example: Download and Use the Model

bash

# 1. Install dependencies
pip install -r requirements.txt

# 2. Download this repository
git clone https://huggingface.co/Ylemnox/llama-toefl-checkbot
cd llama-toefl-checkbot

# 3. Run the web app (make sure to set model paths first)
streamlit run toefl_judge_app.py

Training Details

The model was trained using LoRA with the following configuration:

Configuration available in lora_config.json
Training performed on TOEFL speaking response data
Multiple checkpoints saved during training

Evaluation Results

Evaluation metrics and visualizations are available in:

toefl_evaluations.csv - Detailed metrics
toefl_evaluation_results.png - Performance visualization

License

Please ensure compliance with Meta's Llama license when using this model.

Citation

If you use this model, please cite:

bibtex

@misc{llama-toefl-checkbot,
  author = {Davis Kwak},
  title = {TOEFL Speaking Evaluation Model},
  year = {2025},
  publisher = {Hugging Face},
  howpublished = {\url{https://huggingface.co/Ylemnox/llama-toefl-checkbot}}
}