exp_hp_lr1e-5_r32_a64 – AI Model by ekunish | AlphaNeural AI

You can deploy this model and start earning money today!

ekunish/
exp_hp_lr1e-5_r32_a64

like

0

text-generation

Qwen/Qwen3-4B-Instruct-2507

Views

No views yet

exp_hp_lr1e-5_r32_a64

Model ID: ekunish/exp_hp_lr1e-5_r32_a64

HP grid: lr=1e-5, r=32, alpha=64

Training Configuration

Parameter	Value
Base model	`Qwen/Qwen3-4B-Instruct-2507`
Method	QLoRA (4-bit)
Max sequence length	512
Epochs	1
Learning rate	1e-05
LoRA r	32
LoRA alpha	64
Batch size	2 × 8 = 16

Usage

python

from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel
import torch

base = "Qwen/Qwen3-4B-Instruct-2507"
adapter = "ekunish/exp_hp_lr1e-5_r32_a64"

tokenizer = AutoTokenizer.from_pretrained(base)
model = AutoModelForCausalLM.from_pretrained(
    base,
    torch_dtype=torch.float16,
    device_map="auto",
)
model = PeftModel.from_pretrained(model, adapter)

Training Data

Dataset: data/sft_dedup_pair
License: CC-BY-4.0 (where applicable)

Sources & License

Training Data: u-10bei/structured_data_with_cot_dataset_512_v2, daichira/structured-3k-mix-sft, etc.
Dataset License: Creative Commons Attribution (CC-BY-4.0)
Compliance: Users must comply with both the dataset's attribution requirements and the base model's original terms of use.

Competition

松尾研LLMコミュニティ 2025年度講座メインコンペ (StructEval-T)