Model Card: Math Homework Solver

Model Details

• Base Model: Llama 3.1 (8 Billion parameters)
• Fine-tuning Method: PEFT (Parameter-Efficient Fine-Tuning) with QLoRA
• Quantization: 4-bit quantization for reduced memory usage
• Training Framework: Unsloth, optimized for efficient fine-tuning of large language models
• Training Environment: Google Colab (free tier), NVIDIA T4 GPU (16GB VRAM), 12GB RAM
• Dataset Used: TIGER-Lab/MathInstruct (Yue, X., Qu, X., Zhang, G., Fu, Y., Huang, W., Sun, H., Su, Y., & Chen, W. (2023). MAmmoTH: Building Math Generalist Models through Hybrid Instruction Tuning. arXiv preprint arXiv:2309.05653. ), 560 selected math problems and solutions
• Git Repo: The git repo on my github account is justsomerandomdude264/Homework_Solver_LLM

Capabilities

Getting Started

1. Clone the repo
   git clone https://huggingface.co/justsomerandomdude264/Math_Homework_Solver-Llama3.18B
2. Run inference
   1. This method is recommended as its reliable and accurate:
   python

   1from unsloth import FastLanguageModel 2import torch 3 4# Define Your Question 5question = "Verify that the function y = a cos x + b sin x, where, a, b ∈ R is a solution of the differential equation d2y/dx2 + y=0." # Example Question, You can change it with one of your own 6 7# Load the model 8model, tokenizer = FastLanguageModel.from_pretrained( 9 model_name = "Math_Homework_Solver_Llama318B/model_adapters", # The dir where the repo is cloned or "\\" for root 10 max_seq_length = 2048, 11 dtype = None, 12 load_in_4bit = True, 13 ) 14 15# Set the model in inference model 16FastLanguageModel.for_inference(model) 17 18# QA template 19qa_template = """Question: {} 20Answer: {}""" 21 22# Tokenize inputs 23inputs = tokenizer( 24[ 25 qa_template.format( 26 question, # Question 27 "", # Answer - left blank for generation 28 ) 29], return_tensors = "pt").to("cuda") 30 31# Stream the answer/output of the model 32from transformers import TextStreamer 33text_streamer = TextStreamer(tokenizer) 34_ = model.generate(**inputs, streamer = text_streamer, max_new_tokens = 512)
   1. Another way to run inference is to use the merged adapters (not recommend as it gives inaccurate/different answers):
   python

   1from transformers import LlamaForCausalLM, AutoTokenizer 2 3# Load the model 4model = LlamaForCausalLM.from_pretrained( 5 "justsomerandomdude264/Math_Homework_Solver_Llama318B", 6 device_map="auto" 7) 8 9# Load the tokenizer 10tokenizer = AutoTokenizer.from_pretrained("justsomerandomdude264/Math_Homework_Solver_Llama318B") 11 12# Set the inputs up 13qa_template = """Question: {} 14Answer: {}""" 15 16inputs = tokenizer( 17[ 18 qa_template.format( 19 "Verify that the function y = a cos x + b sin x, where, a, b ∈ R is a solution of the differential equation d2y/dx2 + y=0.", # Question 20 "", # output - leave this blank for generation! 21 ) 22], return_tensors = "pt").to("cuda") 23 24# Do a forward pass 25outputs = model.generate(**inputs, max_new_tokens = 128, use_cache = True) 26raw_output = str(tokenizer.batch_decode(outputs)) 27 28# Formtting the string 29# Removing the list brackets and splitting the string by newline characters 30formatted_string = raw_output.strip("[]").replace("<|begin_of_text|>", "").replace("<|eot_id|>", "").strip("''").split("\\n") 31 32# Print the lines one by one 33for line in formatted_string: 34 print(line)

Citation

BibTeX Citation

APA Citation