You can deploy this model and start earning money today!

kaleido-jean/
q-FrozenLake-v1-4x4-noSlippery

Name: q-FrozenLake-v1-4x4-noSlippery
Availability: InStock

FrozenLake-v1-4x4-no_slippery

q-learning

reinforcement-learning

custom-implementation

model-index

Views

No views yet

Q-Learning Agent playing FrozenLake-v1

This is a trained model of a Q-Learning agent playing FrozenLake-v1 (HF Deep RL Course Unit 2).

Model Details

Algorithm: tabular Q-Learning (from scratch, NumPy Q-table)
Library: Gymnasium + NumPy (no Stable-Baselines3 / no GPU kernels)
Training episodes: 10,000
Eval result: mean_reward = 1.00 +/- 0.00
Non-slippery 4x4 map. Reward is 1 only on reaching the goal.

Usage

python

model = load_from_hub(repo_id="kaleido-jean/q-FrozenLake-v1-4x4-noSlippery", filename="q-learning.pkl")

# Don't forget to check if you need to add additional attributes (is_slippery=False etc)
env = gym.make(model["env_id"])

Training Details

Training Procedure

Training regime: fp64/CPU NumPy table updates (no mixed precision)
Wall-clock train time: 10,000 episodes ≈ 20k eps/s (tqdm wall-clock ~0.5 s)

Training Hardware / Carbon

Tabular Q-Learning (NumPy) is CPU-only. The training node also has an NVIDIA H100 80GB GPU, but it was not used.

Hardware Type: CPU — Intel(R) Xeon(R) Platinum 8470 (node also has NVIDIA H100 80GB HBM3, unused)
Hours used: ~0.00014 h (~0.5 s) (wall-clock training loop only; excludes Hub upload / video encode)
Cloud Provider: PSC Bridges-2 / university HPC cluster
Compute Region: US (Pittsburgh Supercomputing Center)
Carbon Emitted: negligible for this wall-clock (sub-minute CPU job)

Training Hyperparameters

See q-learning.pkl keys: learning_rate, gamma, max_epsilon, min_epsilon, decay_rate, n_training_episodes, max_steps.

Technical Specifications

Compute Infrastructure

Training: single CPU process on Intel Xeon Platinum 8470
GPU: not required / not used (tabular method)