Views
No views yet
google/gemma-3-12b-it for legal obligation extraction from contracts.| Config | Value |
|---|---|
| Quantization | 4-bit NF4 + double quant |
| LoRA | r=64, alpha=128, dropout=0.05 |
| Target Modules | q/k/v/o_proj, gate/up/down_proj |
| Learning rate | 2e-4 (cosine schedule) |
| Epochs | 3 |
| Effective batch | 32 (4 x 8 grad accum) |
| Max seq length | 2048 |
| Precision | bf16 |
| Dataset | CUAD (obligation-filtered) |
| Metric | Baseline | Fine-Tuned | Improvement |
|---|---|---|---|
| Valid JSON | 100.0% | 100.0% | +0.0% |
| Schema Valid | 100.0% | 100.0% | +0.0% |
| Deontic Acc | 58.0% | 77.0% | +19.0% |
| ROUGE-L | 55.6% | 84.5% | +28.9% |
| Field Comp | 100.0% | 100.0% | +0.0% |
1from transformers import AutoTokenizer, AutoModelForCausalLM
2
3model = AutoModelForCausalLM.from_pretrained("coder123d/lexis-gemma3-12b-obligation-generator")
4tokenizer = AutoTokenizer.from_pretrained("coder123d/lexis-gemma3-12b-obligation-generator")
5
6messages = [{"role": "user", "content": "Extract the legal obligation from: ...\n\nReturn JSON."}]
7prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
8inputs = tokenizer(prompt, return_tensors='pt')
9output = model.generate(**inputs, max_new_tokens=512)
10print(tokenizer.decode(output[0], skip_special_tokens=True))