import torch
from unsloth import FastLanguageModel
from transformers import AutoModelForCausalLM, AutoTokenizer

model, tokenizer = FastLanguageModel.from_pretrained(model_name="amjad-awad/mistral-7b-instruct-v0.2-bnb-4bit-EngSaf-96K-lr-1e5",max_seq_length=2048,load_in_4bit=True)

model = FastLanguageModel.get_peft_model(
    model,
    r=16,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],
    lora_alpha=16,
    lora_dropout=0,
    bias="none",
    use_gradient_checkpointing=True,
    random_state=3407,
)

user_content = (
    "Provide both a score and a rationale by evaluating the student's answer strictly within the mark scheme range, "
    "grading based on how well it meets the question's requirements by comparing the student answer to the reference answer.\n"
    "Question: What is photosynthesis?\n"
    "Reference Answer: Photosynthesis is the process by which green plants and some other organisms use sunlight to synthesize nutrients from carbon dioxide and water. It generally involves the green pigment chlorophyll and generates oxygen as a by-product.\n"
    "Student Answer: Photosynthesis is how plants make their food using sunlight and carbon dioxide. It also gives off oxygen.\n"
    "Mark Scheme: {'1':'Mentions use of sunlight', '2':'Mentions carbon dioxide and water', '3':'Mentions production of oxygen', '4':'Explains synthesis of nutrients or food', '5':'Mentions chlorophyll or green pigment'}"
)

user = [
    {"role":"system", "content": "You are a grading assistant. Evaluate student answers based on the mark scheme. Respond only in JSON format with keys 'score' (int) and 'rationale' (string)."},
    {"role":"user", "content": user_content},
      ]

inputs = tokenizer.apply_chat_template(user, tokenize=True, add_generation_prompt=True, return_tensors="pt", return_dict=True).to(model.device)
generated_ids = model.generate(**inputs, max_new_tokens=128, temperature=0.2, top_k=5, do_sample=False)[0]
new_generated_ids = generated_ids[inputs["input_ids"].shape[1]:]
generated_text = tokenizer.decode(new_generated_ids, skip_special_tokens=True)

print(generated_text)

Results:

{"score": 5, "rationale": "Your answer is correct. You have accurately described the process of photosynthesis, mentioning the use of sunlight, carbon dioxide, and water, and the production of food and oxygen as by-products. Keep up the good work!"}

Training hyperparameters

The following hyperparameters were used during training:

per_device_train_batch_size:1
per_device_eval_batch_size:1
gradient_accumulation_steps:8
eval_strategy:"steps"
save_strategy:"steps"
eval_steps:10
logging_dir:"./logs"
logging_steps:10
save_total_limit:1
learning_rate:1e-5
warmup_steps:100
weight_decay:0.01
num_train_epochs:3
load_best_model_at_end:True
lr_scheduler_type:"cosine"
metric_for_best_model:"eval_loss"
greater_is_better:False

Training results

Step	Training Loss	Validation Loss
10	3.249500	3.304678
20	3.251500	3.268980
30	3.217000	3.200722
40	3.075900	3.100004
50	3.016900	2.967428
60	2.841200	2.804593
70	2.707600	2.622378
80	2.545500	2.426775
90	2.301500	2.212558
100	2.071500	1.997747
110	1.893100	1.798315
120	1.705800	1.592137
130	1.544700	1.481935
140	1.467700	1.415517
150	1.404500	1.377060
160	1.318000	1.357312
170	1.401800	1.349880
180	1.361200	1.347993

Framework versions

Transformers 4.51.3
Pytorch 2.7.0
Datasets 3.6.0
Unsloth 2025.5.6