You can deploy this model and start earning money today!

StevanLS/
ddpg-panda-reach-100

Name: ddpg-panda-reach-100
Availability: InStock

stable-baselines3

PandaReachJointsDense-v3

deep-reinforcement-learning

reinforcement-learning

Views

No views yet

DDPG Panda Reach Model

This is a DDPG (Deep Deterministic Policy Gradient) model trained to control a Franka Emika Panda robot arm in a reaching task using dense rewards. The model was trained using Stable-Baselines3 with Hindsight Experience Replay (HER).

Task Description

In this task, a 7-DOF Panda robotic arm must reach a randomly positioned target in 3D space. The environment provides dense rewards based on the distance between the end-effector and the target position. The task is considered successful when the end-effector reaches within a small threshold distance of the target.

Training Details

Environment: PandaReachJointsDense-v3 from panda-gym
Algorithm: DDPG with HER
Policy: MultiInputPolicy
Training Steps: 100,000
Framework: Stable-Baselines3
Training Monitoring: Weights & Biases

Hyperparameters

python

{
    "policy": "MultiInputPolicy",
    "replay_buffer_class": "HerReplayBuffer",
    "tensorboard_log": True,
    "verbose": 1,
    "total_timesteps": 100000
}

Usage

python

import gymnasium as gym
import panda_gym
from stable_baselines3 import DDPG

# Create environment
env = gym.make("PandaReachJointsDense-v3", render_mode="human")

# Load the trained model
model = DDPG.load("StevanLS/ddpg-panda-reach-100")

# Run the model
obs, _ = env.reset()
while True:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, truncated, info = env.step(action)
    if done or truncated:
        obs, _ = env.reset()

Limitations

The model is trained specifically for the reaching task and may not generalize to other manipulation tasks
Performance may vary depending on the random target positions
The model uses dense rewards, which might not be available in real-world scenarios

Author

StevanLS

Citations

bibtex

@article{raffin2021stable,
    title={Stable-baselines3: Reliable reinforcement learning implementations},
    author={Raffin, Antonin and Hill, Ashley and Gleave, Adam and Kanervisto, Anssi and Ernestus, Maximilian and Dormann, Noah},
    journal={Journal of Machine Learning Research},
    year={2021}
}

@article{gallouedec2021pandagym,
    title={panda-gym: Open-Source Goal-Conditioned Environments for Robotic Learning},
    author={Gallou{\'e}dec, Quentin and Cazin, Nicolas and Dellandr{\'e}a, Emmanuel and Chen, Liming},
    journal={arXiv preprint arXiv:2106.13687},
    year={2021}
}

@article{gymatorium2023,
    author={Farama Foundation},
    title={Gymnasium},
    year={2023},
    journal={GitHub repository},
    publisher={GitHub},
    url={https://github.com/Farama-Foundation/Gymnasium}
}