You can deploy this model and start earning money today!

yujackein/
onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-seq10k-gbs4-1epoch

Name: onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-seq10k-gbs4-1epoch
Availability: InStock

peft

safetensors

recommendation

generative-recommendation

OpenOneRec/OneReason-8B-pretrain-competition

adapter

Views

No views yet

OneReason-8B LoRA: Item32K / User75 / Rec50 / WorldClean1601 / Seq10K / GBS4 / 1 Epoch

This is an experimental LoRA adapter for OpenOneRec/OneReason-8B-pretrain-competition, trained for the OneReason recommendation competition.

Data recipe

The source mixture is the Item32K / User75 / Rec50 / WorldClean1601 recipe:

Category	Source rows	Source rendered tokens
R0 perception	32,000	5,801,468
R2 evolution	24,636	119,130,968
R3 recommendation	24,134	34,445,219
Clean C-Eval world data	1,601	195,253
Total	82,371	159,572,908

For the 10,000-token run, 15 R2_evolution rows longer than the cutoff were removed because head truncation would eliminate all assistant supervision. The resulting training set has 82,356 rows and 159,413,369 rendered tokens. The exact four-rank packing plan consumes 99.0516% of these tokens in one safe pass. All assistant loss weights were normalized to 1.0.

Training

Base model: OneReason-8B pretrain competition release
Method: LoRA (r=32, alpha=32, dropout 0.05)
Target modules: q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Precision: BF16
Context length: 10,000
Hardware: 4 x NVIDIA A800 80GB
Micro batch size: 1 per GPU
Global batch size: 4 (no gradient accumulation)
Learning rate: 2e-4, cosine schedule, 3% warmup
Training length: one safe packing pass, 5,030 optimizer updates
Seed: 42
Final logged loss: 0.7464
Completed: 2026-08-12 23:46 CST

Adapter SHA-256: ec65db55748eeb8181591b53ba1d2eb7f3dfea702e5e32552f93e6198f45a776

Closest comparison

The closest existing run is yujackein/onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-step323, the one-packing-pass checkpoint from the 32,768-context r32/a32 LR2e-4 run.

This is not a strict sequence-length-only ablation. Relative to that step-323 checkpoint, this run also changes global batch size from 16 to 4, uses a standalone one-epoch cosine schedule rather than the first half of a two-epoch schedule, and removes 15 overlength rows. The data recipe, base model, LoRA configuration, optimizer, peak learning rate, precision, and seed are otherwise the same.

Usage

python

import torch
from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base_id = "OpenOneRec/OneReason-8B-pretrain-competition"
adapter_id = "yujackein/onereason-8b-lora-item32k-user75-rec50-worldclean1601-all1-lr2e4-r32a32-seq10k-gbs4-1epoch"

tokenizer = AutoTokenizer.from_pretrained(adapter_id, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    base_id,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)
model = PeftModel.from_pretrained(base_model, adapter_id)

Evaluation

Formal evaluation results have not yet been added.