You can deploy this model and start earning money today!

IntelliGrow/
FetchPickAndPlace-v4

Name: FetchPickAndPlace-v4
Rating: 2.5 (90 reviews)

stable-baselines3

FetchPickAndPlace-v4

deep-reinforcement-learning

reinforcement-learning

model-index

Views

No views yet

SAC + HER Agent for FetchPickAndPlace-v4

Model Overview

This repository contains a Soft Actor-Critic (SAC) agent trained with Hindsight Experience Replay (HER) on the FetchPickAndPlace-v4 environment from gymnasium-robotics. The agent learns to pick and place objects using sparse or dense rewards, and is suitable for robotic manipulation research.

Algorithm: Soft Actor-Critic (SAC)
Replay Buffer: Hindsight Experience Replay (HER)
Environment: FetchPickAndPlace-v4 (gymnasium-robotics)
Framework: Stable Baselines3

Training Details

Total Timesteps: 500,000
Evaluation Frequency: Every 2,000 steps (15 episodes per eval)
Checkpoint Frequency: Every 50,000 steps (model + replay buffer)
Seed: 42
Dense Shaping: False (can be enabled with wrapper)
Device: CUDA if available, otherwise auto

Hyperparameters

Parameter	Value
Algorithm	SAC
Policy	MultiInputPolicy
Replay Buffer	HER
n_sampled_goal	4
goal_selection_strategy	future
Batch Size	512
Buffer Size	1,000,000
Learning Rate	1e-3
Gamma	0.95
Tau	0.05
Entropy Coefficient	auto
Train Frequency	1 step
Gradient Steps	1
Tensorboard Log	logs_pnp_sac_her/tb
Seed	42
Device	CUDA/Auto
Dense Shaping	False (default)

Files

sac_her_pnp.zip: Final trained SAC model
ckpt_sac_her_250000_steps.zip: Latest checkpoint
replay_buffer.pkl: Replay buffer for continued training
replay.mp4: Replay video of agent performance (manual generation recommended)
README.md: This model card

Usage

To load and use the model for inference:

python

from stable_baselines3 import SAC
import gymnasium as gym
import gymnasium_robotics

env = gym.make("FetchPickAndPlace-v4", render_mode="rgb_array")
model = SAC.load("path/to/sac_her_pnp.zip", env=env)

obs, info = env.reset()
done = False
while not done:
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, truncated, info = env.step(action)
    env.render()

Evaluation

To evaluate the agent over multiple episodes:

python

from stable_baselines3 import SAC
import gymnasium as gym
import gymnasium_robotics

env = gym.make("FetchPickAndPlace-v4", render_mode="human")
model = SAC.load("path/to/sac_her_pnp.zip", env=env)

num_episodes = 10
for ep in range(num_episodes):
    obs, info = env.reset()
    done = False
    truncated = False
    episode_reward = 0
    while not (done or truncated):
        action, _ = model.predict(obs, deterministic=True)
        obs, reward, done, truncated, info = env.step(action)
        env.render()
        episode_reward += reward
    print(f"Episode {ep+1} reward: {episode_reward}")
env.close()

Replay Video

If replay.mp4 is not present, you can manually generate it:

python

import gymnasium as gym
import gymnasium_robotics
from stable_baselines3 import SAC
import moviepy.editor as mpy

env = gym.make("FetchPickAndPlace-v4", render_mode="rgb_array")
model = SAC.load("path/to/sac_her_pnp.zip", env=env)

frames = []
obs, info = env.reset()
done = False
truncated = False
step = 0
max_steps = 1000

while not (done or truncated) and step < max_steps:
    frame = env.render()
    frames.append(frame)
    action, _ = model.predict(obs, deterministic=True)
    obs, reward, done, truncated, info = env.step(action)
    step += 1

env.close()
clip = mpy.ImageSequenceClip(frames, fps=30)
clip.write_videofile("replay.mp4", codec="libx264")

Continued Training

To continue training from a checkpoint:

python

from stable_baselines3 import SAC
import gymnasium as gym
import gymnasium_robotics

env = gym.make("FetchPickAndPlace-v4", render_mode=None)
model = SAC.load("logs_pnp_sac_her/ckpt_sac_her_250000_steps.zip", env=env)
model.learn(total_timesteps=500_000, reset_num_timesteps=False)

Citation

If you use this model, please cite:

@misc{IntelliGrow_FetchPickAndPlace_SAC_HER,
  title={SAC + HER Agent for FetchPickAndPlace-v4},
  author={IntelliGrow},
  year={2025},
  howpublished={Hugging Face Hub},
  url={https://huggingface.co/IntelliGrow/FetchPickAndPlace-v4}
}

License

MIT License

Contact: For questions or issues, open an issue on the Hugging Face repository.