You can deploy this model and start earning money today!

LTU-AI/
hdppo-Ant-v5

Name: hdppo-Ant-v5
Availability: InStock

hd-ppo

Ant-v5

deep-reinforcement-learning

reinforcement-learning

hyperdimensional-computing

fractional-power-encoding

LTU-AI

model-index

Views

No views yet

Hybrid-HD-PPO Agent playing Ant-v5

This is a trained Hybrid-HD-PPO (Hyperdimensional Proximal Policy Optimization) agent playing Ant-v5 using gradient-adaptive Fractional Power Encoding (FPE) with a prune-and-fine-tune pipeline.

Published by LTU-AI.

Pipeline

Train a teacher at D=512 with gradient-adaptive single-beta FPE.
Prune by actor-weight importance through D=512 → 128 → 64.
Fine-tune each pruned checkpoint with PPO.

Published checkpoint: seed 42, compact D=64 model (held-out eval mean reward 4351.56 ± 102.60).

Usage

Install dependencies:

pip install -r requirements.txt

Evaluate the local checkpoint:

python enjoy.py --weights hdppo-Ant-v5/weights.npz --episodes 10

Render episodes:

python enjoy.py --weights hdppo-Ant-v5/weights.npz --render --episodes 3

Record a replay video:

python record_video.py --weights hdppo-Ant-v5/weights.npz --output replay.mp4

Load from Hugging Face Hub:

python enjoy.py --weights LTU-AI/hdppo-Ant-v5 --episodes 10

Training pipeline

Reproduce the teacher → prune → fine-tune workflow:

python run_prune_finetune_5seed.py

Hyperparameters

python

{
    "env": "Ant-v5",
    "algo": "Hybrid-HD-PPO (HD actor + MLP critic, gradient-adaptive FPE)",
    "teacher_D": 512,
    "pruned_D": 64,
    "beta": 0.8,
    "timesteps_per_stage": 1000000,
    "seed": 42
}

Environment Arguments

python

{
    "render_mode": "rgb_array"
}

Model files

File	Description
`hdppo-Ant-v5/weights.npz`	Published actor (+ critic if HD) and FPE encoder (D=64)
`hdppo-Ant-v5/weights_D512_teacher.npz`	Teacher checkpoint (D=512)
`replay.mp4`	Sample rollout video from the published min-D checkpoint
`results.json`	Evaluation summary for the published checkpoint
`results_D512_teacher.json`	Evaluation summary for the teacher
`config.yml`	Training hyperparameters
`train_hdppo.py` / training modules	Self-contained training code

Citation

If you use this model, please cite the HD-PPO / Hybrid-HD-PPO work.