QLoRA (rank 16) fine-tune of google/gemma-2-2b on the same synthetic legal/financial instruction dataset used for Probabilistr/slm-125m-finetuned (grounded QA/RAFT, summarization, extraction-to-JSON, and plain-English rewrite tasks generated with gpt-4o-mini from case-law and SEC-filing passages). See the slm-125m project's Instructions_for_finetuning.txt for the full data pipeline.
This repo contains a LoRA adapter only -- load with peft.PeftModel.from_pretrained on top of the base model, or merge with merge_and_unload().
Prompt format
Gemma's native turn format, with the system prompt folded into the user turn (Gemma has no separate system role):
<bos><start_of_turn>user
You are a legal/financial assistant. Answer strictly using the information in the provided context. If the answer is not stated in the context, say "Not stated in the context."
Context:
{context}
Instruction: {instruction}<end_of_turn>
<start_of_turn>model
Generation should stop on <end_of_turn>, not just <eos>.
