Base model: Qwen/Qwen2.5-0.5B-Instruct
Training data: 46 (prompt, action_json) pairs collected by rolling out the heuristic policy across the 6 environment tasks
Loss: causal LM (next-token prediction over the action JSON suffix)
Optimizer: AdamW (lr 2e-5)
Epochs: 1
Sequence length: 128
Hardware: CPU (smoke run); for production use, retrain on GPU with the PPO recipe below
Loss curve: 3.20 → 0.07 across 46 steps

Training script: training/train_sft_simple.py

Inference

python

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("M134pra/neon-syndicate-qwen25-sft")
model = AutoModelForCausalLM.from_pretrained("M134pra/neon-syndicate-qwen25-sft")

prompt = "You are a strategic planner in Neon Syndicate. Return only one JSON action.\n..."
inputs = tokenizer(prompt, return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=64, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[-1]:], skip_special_tokens=True))

Companion Resources

Space (live environment API + landing page): https://huggingface.co/spaces/M134pra/openenvhack
Source code & training scripts: same Space repository
Reward / loss curves and eval metrics: artifacts/ in the Space repo

Limitations

This checkpoint is a CPU-friendly smoke run intended to demonstrate the training pipeline end-to-end. It under-performs the heuristic baseline on average task score because it was trained on only 46 examples for 1 epoch. For competitive numbers, run the PPO recipe in training/train_trl_ppo.py on a GPU.