agent-2048-game-qwen-7b-8k-ds

Model Description

• Base Model: Qwen-7B-Instruct
• Training Approach: Group Relative Policy Optimization (GRPO)
• Training Dataset: 8,000 carefully curated game states
• Hardware Used: Single RTX 4090 (24GB)
• Training Time: ~10 hours
• Framework: Implemented using trl library and accelerated by Unsloth

Training Configuration

• Learning Rate: 4e-5 (optimized after extensive testing)
• LoRA Rank: 16
• Max Sequence Length: 1000 tokens
• Batch Size: 1 (with gradient accumulation steps of 4)
• Optimizer: paged_adamw_8bit

Intended Use

1. Analyzing the current board state
2. Planning strategic moves
3. Maximizing score and achieving high-value tiles
4. Maintaining efficient board organization

Training Data

• Simulated gameplay for realistic board states
• Custom difficulty scoring system
• 5-level difficulty classification
• Balanced sampling across difficulty levels
• Parallel processing for efficient generation

Training Approach

Reward System

1. Density Reward: Encourages efficient tile merging and space utilization
2. Highest Tile Reward: Incentivizes creation of high-value tiles
3. Survival Reward: Promotes moves that maintain game continuity
4. Format Compliance: Ensures proper response structure

Optimization

• Utilized Unsloth for 2x faster fine-tuning
• 4-bit quantization for efficient training
• Implemented efficient LoRA adaptation

Performance and Limitations

Strengths

• Strong strategic planning capabilities
• Efficient tile merging and space management
• Consistent high-score achievement
• Structured decision-making process

Limitations

• Performance may vary with random seeds
• Success not guaranteed due to game's inherent randomness
• Model requires specific input formatting

Example Usage

Citation

Author

Acknowledgments

License