LoRA adapter trained on 40 Claude Sonnet 4.5 trajectories from the Hospitality RL Environment, a 116-task hot-pot-restaurant simulation built for the 2026 OpenEnv Challenge (Meta PyTorch × Hugging Face × Unsloth).

TL;DR — This is a null-result checkpoint

Honest evaluation finding: this checkpoint is behaviorally identical to the base model on a stratified 20-task held-out eval (both score +0.101, with identical per-task rewards). We release it as a documented null-result artifact — the training pipeline works end-to-end (LoRA applies cleanly, losses decrease monotonically), but the signal the environment rewards does not exist in 40 imitation trajectories evaluated against a terminal-only reward.

Full diagnosis including a cross-family null check (Qwen 7B/14B, Llama 3.1 8B, Gemma 2 9B — all clustered in a 0.056-wide band) is in the repo README.

Metric	Base (Qwen 2.5 7B)	This adapter (v1)	Δ
Stratified eval mean	+0.101	+0.101	0.000
Mean turns	6.10	6.10	0
Claude Sonnet 4.5 (ceiling)	+0.314	—	—

Training recipe

Base model: unsloth/Qwen2.5-7B-Instruct-bnb-4bit
LoRA: r=16, α=32, dropout=0.0, bias=none
Data: 40 Claude Sonnet 4.5 trajectories with reward ≥ 0.2, drawn from a stratified train/eval split (per-category representation preserved; see sft_data/stratified_manifest.json in the env repo)
Training: 3 epochs, 18 steps total, LR 2e-4 (cosine), warmup 0.05, batch size 1
Final train loss: 0.7559
Hardware: single H100 (Lambda Cloud), ~30 min
Framework: Unsloth + TRL SFTTrainer

Intended use

Research — reproducing the null-result diagnosis in the OpenEnv submission
Env iteration — as a reference "no-signal" baseline when testing changes to reward surface, data scale, or policy structure
Not recommended for production — this adapter has zero measured improvement over the base model on the target distribution

Usage

python

from peft import PeftModel
from transformers import AutoModelForCausalLM, AutoTokenizer

base = "unsloth/Qwen2.5-7B-Instruct-bnb-4bit"
adapter = "binleiwang/qwen2.5-7b-hospitality-sft"

model = AutoModelForCausalLM.from_pretrained(base, device_map="auto")
model = PeftModel.from_pretrained(model, adapter)
tokenizer = AutoTokenizer.from_pretrained(base)

For running this adapter against the Hospitality Env end-to-end (multi-turn rollouts with tool calls), see eval_v1_stratified.py in the env repo.

Training curve and alternative recipe

v1 (this release): 3 epochs, final loss 0.7559, eval +0.101
v3 (not released): 6 epochs, final loss 0.0587 (12× lower than v1), eval +0.104

Driving training loss 12× lower moved eval +0.003 — empirical evidence that the optimization landscape and the reward landscape are on different manifolds. This informed the decision to ship v1 as the canonical "safe conservative" recipe rather than v3.

Evaluation protocol

All evaluations use:

Greedy decoding (do_sample=False)
Max 8 turns per task
Same 20-task stratified held-out split across all baselines and trained models
Deterministic scoring via ACTION assertions (required tool calls) + ENV_ASSERTION (DB state checks), no LLM-as-judge

Environmental impact

Hardware: 1× NVIDIA H100 (80GB)
Compute Provider: Lambda Cloud
Training time: ~30 minutes

Citation

bibtex

@misc{wang2026hospitality,
  title  = {Hospitality RL Environment: A Hot Pot Restaurant Simulation},
  author = {Binlei Wang},
  year   = {2026},
  howpublished = {OpenEnv Challenge submission}
}

License

MIT — adapter and model card. Base model (unsloth/Qwen2.5-7B-Instruct-bnb-4bit) retains its own license.