SagaLM-slm1 – AI Model by venkateshchsagalm | AlphaNeural AI

You can deploy this model and start earning money today!

venkateshchsagalm/
SagaLM-slm1

like

0

Qwen/Qwen2.5-3B-Instruct

Views

No views yet

SagaLM-slm1 (LoRA adapter)

LoRA adapter for SagaLM-slm1, fine-tuned on Qwen/Qwen2.5-3B-Instruct.

Training Data (12 sources, long-form filtered)

OpenHermes 2.5, UltraChat 200k, OpenOrca, MetaMathQA,
Function Calling (hypervariance), Open-Platypus,
Glaive Code Assistant, Dolly 15k, ShareGPT GPT-4,
Alpaca-GPT4, SlimOrca, MathInstruct

Only samples with detailed, long assistant answers were kept, training the model to always respond comprehensively.

Identity Reinforcement

5 identity-focused training samples were injected to ensure the model consistently identifies as SagaLM (not Qwen or any other model).

LoRA Configuration

r = 16
lora_alpha = 32
lora_dropout = 0.05
target_modules = ['up_proj', 'q_proj', 'down_proj', 'k_proj', 'gate_proj', 'v_proj', 'o_proj']

How to use

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = AutoModelForCausalLM.from_pretrained("Qwen/Qwen2.5-3B-Instruct", device_map="auto")
model = PeftModel.from_pretrained(base, "venkateshchsagalm/SagaLM-slm1")
tokenizer = AutoTokenizer.from_pretrained("venkateshchsagalm/SagaLM-slm1")