You can deploy this model and start earning money today!

divisee/
qwen2.5-7b-ruslit-classic-lora

Name: qwen2.5-7b-ruslit-classic-lora
Availability: InStock

Qwen/Qwen2.5-7B-Instruct

finetune

endpoints_compatible

Views

No views yet

Qwen2.5-7B RusLit Classic LoRA

LoRA-адаптер для модели Qwen/Qwen2.5-7B-Instruct, дообученный на корпусе художественных русскоязычных текстов (RusLit) в стиле классической литературы.

Модель адаптирована для генерации художественной прозы с сохранением лексики, темпа и стилистики русской классики.

📌 Model Details

Base model: Qwen/Qwen2.5-7B-Instruct
Adaptation method: PEFT / QLoRA (Unsloth)
Task: Style Transfer / Creative Text Generation
Language: Russian
Training type: Supervised Fine-Tuning (SFT)

📊 Training Setup

Dataset

Name: RusLit
Task: Continue literary text in classical Russian style
Train samples: 1,294
Eval samples: 157

Hardware

GPU: 1× H100 / A100
Framework: Unsloth (2× faster fine-tuning)

Training Configuration

Parameter	Value
Epochs	3
Batch size	2
Gradient accumulation	4
Effective batch size	8
Max sequence length	2048
Learning rate	2e-4
Optimizer	adamw_8bit
Scheduler	cosine
Precision	FP16 / BF16

LoRA Configuration

Parameter	Value
r	16
alpha	16
dropout	0.05
Target modules	q,k,v,o,gate,up,down

Trainable params: 40,370,176
Total params: 7,655,986,688
Trainable share: 0.53%

📈 Training Results

Training was performed with early stopping.

Loss Dynamics

Step	Train Loss	Eval Loss
25	2.1428	2.2432
50	2.2067	2.2225
100	2.0872	2.2045
150	2.0864	2.1903
225	1.9405	2.2067

Best checkpoint: checkpoint-150
Best eval loss: 2.1903

The model converged early and reached a local optimum around step 150. Further training did not improve validation loss and showed signs of overfitting.

🚀 Usage

Installation

pip install -U transformers peft accelerate bitsandbytes

python

from unsloth import FastLanguageModel
import torch

base_model = "Qwen/Qwen2.5-7B-Instruct"
adapter_model = "divisee/qwen2.5-7b-ruslit-classic-lora"

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = base_model,
    max_seq_length = 2048,
    dtype = torch.float16,
    load_in_4bit = True,
)

model = FastLanguageModel.get_peft_model(model)

model.load_adapter(adapter_model, adapter_name="ruslit")
model.set_adapter("ruslit")

model.eval()

python

prompt = """
Ты пишешь в стиле русской классики.

Продолжи художественный текст:

Над вечерней рекой поднимался туман и ...
"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)

outputs = model.generate(
    **inputs,
    max_new_tokens=300,
    temperature=0.8,
    top_p=0.9,
    do_sample=True,
)

print(tokenizer.decode(outputs[0], skip_special_tokens=True))

@misc{divisee2026ruslit, title={Qwen2.5-7B RusLit Classic LoRA}, author={divisee}, year={2026}, url={https://huggingface.co/divisee/qwen2.5-7b-ruslit-classic-lora} }