A lightweight helper model that turns Git diffs into Conventional Commit–style messages. It outputs strict JSON with a short title (≤ 65 chars) and up to 3 bullets, so your CLI/agents can parse it deterministically.

Model Details

Model Description

Purpose: Summarize git diff patches into concise, Conventional Commit–compliant titles with optional bullets.
I/O format:
- Input: prompt containing the diff (plain text).
- Output: JSON object: {"title": "...", "bullets": ["...", "..."]}.
Model type: LoRA adapter for causal LM (text generation)
Language(s): English (commit message conventions)
Finetuned from: unsloth/LFM2-350M-unsloth-bnb-4bit (4-bit quantized base, trained with QLoRA)

Model Sources

Repository: This model card + adapter on the Hub under ethanke/lfm2_350m_commit_diff_summarizer

Uses

Direct Use

Convert patch diffs into Conventional Commit messages for PR titles, commits, and changelogs.
Provide human-readable summaries in agent UIs with guaranteed JSON structure.

Recommendations

Enforce JSON validation; if invalid, retry with a JSON-repair prompt.
Keep a regex gate for Conventional Commit titles in your pipeline.

How to Get Started

python

from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel
import torch, json

BASE = "unsloth/LFM2-350M-unsloth-bnb-4bit"
ADAPTER = "ethanke/lfm2_350m_commit_diff_summarizer"  # replace with your repo id

bnb = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type="nf4",
                         bnb_4bit_use_double_quant=True, bnb_4bit_compute_dtype=torch.float16)

tok = AutoTokenizer.from_pretrained(BASE, use_fast=True)
mdl = AutoModelForCausalLM.from_pretrained(BASE, quantization_config=bnb, device_map="auto")
mdl = PeftModel.from_pretrained(mdl, ADAPTER)

diff = "...your git diff text..."
prompt = (
  "You are a commit message summarizer.\n"
  "Return a concise JSON object with fields 'title' (<=65 chars) and 'bullets' (0-3 items).\n"
  "Follow the Conventional Commit style for the title.\n\n"
  "### DIFF\n" + diff + "\n\n### OUTPUT JSON\n"
)

inputs = tok(prompt, return_tensors="pt").to(mdl.device)
with torch.no_grad():
    out = mdl.generate(**inputs, max_new_tokens=200, do_sample=False)
text = tok.decode(out[0], skip_special_tokens=True)

# naive JSON extraction
js = text[text.rfind("{"): text.rfind("}")+1]
obj = json.loads(js)
print(obj)

Training Details

Training Data

Dataset: Maxscha/commitbench (diff → commit message).
Filtering: kept only samples whose first non-empty line of the message matches Conventional Commits: ^(feat|fix|docs|style|refactor|perf|test|build|ci|chore|revert)($[^)]+$)?(!)?:\s.+$
Note: The dataset card indicates non-commercial licensing. Confirm before commercial deployment.

Training Procedure

Method: Supervised fine-tuning (SFT) with TRL SFTTrainer + QLoRA (PEFT).
Prompting: Instruction + ### DIFF + ### OUTPUT JSON target (title/bullets).
Precision: fp16 compute on 4-bit base.
Hyperparameters (v0.1):
- max_length=2048, per_device_train_batch_size=2, grad_accum=4
- lr=2e-4, scheduler=cosine, warmup_ratio=0.03
- epochs=1 over capped subset
- LoRA: r=16, alpha=32, dropout=0.05, targets: q/k/v/o + MLP proj

Evaluation

Validation: filtered split from CommitBench.
Metrics (example run):
- eval_loss ≈ 1.18 → perplexity ≈ 3.26
- eval_mean_token_accuracy ≈ 0.77
- Suggested task metrics: JSON validity rate, CC-title compliance, title length ≤ 65 chars, bullets ≤ 3.

Environmental Impact

Hardware: 1× NVIDIA GTX 3060 12 GB (local)
Hours used: ~2 h (prototype)

Technical Specifications

Architecture: LFM2-350M (decoder-only) + LoRA adapter
Libraries: transformers, trl, peft, bitsandbytes, datasets, unsloth

Contact

Open an issue on the Hub repo or message ethanke on Hugging Face.

Framework versions

PEFT 0.17.1
TRL (SFTTrainer)
Transformers (recent version)

ethanker/lfm2_350m_commit_diff_summarizer

lfm2_350m_commit_diff_summarizer (LoRA)