Property	Value
Base model	Qwen2.5-1.5B-Instruct
Fine-tuning	LoRA / PEFT
LoRA rank	16
LoRA alpha	32
LoRA dropout	0.05
Language	English
Training objective	Reward-weighted supervised fine-tuning

Training

The model was trained on content–title pairs.

Instead of treating every training example equally, examples were weighted according to their associated engagement score. Higher-scoring examples therefore contribute more strongly to the training loss.

This approach is intended to encourage title characteristics associated with higher engagement while retaining the broader writing patterns present in the training data.

The model was trained for 2 epochs on a randomly sampled 25,000-example subset of the larger dataset.

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model = AutoModelForCausalLM.from_pretrained(
    "csankalp21/headlinegpt",
    torch_dtype=torch.float16,
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained(
    "csankalp21/headlinegpt"
)

messages = [
    {
        "role": "system",
        "content": "You are an expert at writing highly engaging titles."
    },
    {
        "role": "user",
        "content": (
            "Generate a high-engagement title for the following content:\n\n"
            "<your content here>"
        )
    }
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer(
    text,
    return_tensors="pt"
).to(model.device)

with torch.no_grad():
    output = model.generate(
        **inputs,
        max_new_tokens=40,
        temperature=0.7,
        do_sample=True,
        top_p=0.9,
        repetition_penalty=1.1
    )

generated_tokens = output[0][inputs["input_ids"].shape[1]:]

print(
    tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True
    )
)