import torch
from peft import PeftModel
from transformers import AutoModelForSeq2SeqLM, AutoTokenizer, GenerationConfig

base_model_id = "IlyaGusev/rut5_base_sum_gazeta"
adapter_id = "YOUR_USERNAME/YOUR_MODEL_NAME"

tokenizer = AutoTokenizer.from_pretrained(adapter_id)
base_model = AutoModelForSeq2SeqLM.from_pretrained(base_model_id)
model = PeftModel.from_pretrained(base_model, adapter_id)
model.eval()
generation_config = GenerationConfig.from_pretrained(adapter_id)

prompt = """Суммаризируй новость в стиле добрых новостей. Пиши 2-4 предложения, тёпло и очень оптимистично! Подчеркни помощь, пользу, достижение, поддержку или конструктивное изменение, если это есть в новости. Используй только факты из новости и не добавляй оценок.

Заголовок:
Заголовок новости

Новость:
Текст новости"""

inputs = tokenizer(prompt, return_tensors="pt", max_length=512, truncation=True)
with torch.no_grad():
    output_ids = model.generate(
        **inputs,
        generation_config=generation_config,
    )

print(tokenizer.decode(output_ids[0], skip_special_tokens=True))

For reproducible output, use the prompt exactly as shown and load generation_config.json from this adapter repository. The generation uses beam search with do_sample=False, so sampling parameters such as temperature and top_p should not be added.

Training

Train examples: 1669
Validation examples: 186
LoRA rank: 8
LoRA alpha: 16
LoRA dropout: 0.05
Target modules: q, v
Learning rate: 2e-4
Epochs: 5
Trainable parameters: 884736 (0.36%)

Limitations

The adapter can hallucinate or confuse names, roles, organizations, numbers, locations, sports and event results. Generated summaries must be fact-checked against the source article. The training targets were generated automatically and may contain overly positive or evaluative wording.

tanyamr/rut5-good-news-lora-v5

RuT5 Good News LoRA v5

Usage

Training

Limitations

tanyamr/
rut5-good-news-lora-v5