mirror of
https://github.com/wassname/pytorch-soft-actor-critic.git
synced 2026-06-27 18:06:10 +08:00
Pranjal Tandon
2.2 KiB
2.2 KiB
Description
Reimplementation of Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor.
Contributions are welcome. If you find any mistake (very likely) or know how to make it more stable, don't hesitate to send a pull request.
Requirements
Run
For SAC :
python main.py --env-name Humanoid-v2 --scale_R 20
For SAC (Hard Update):
python main.py --env-name Humanoid-v2 --scale_R 20 --tau 1 --target_update_interval 1000
For SAC (Deterministic, Hard Update):
python main.py --env-name Humanoid-v2 --scale_R 20 --deterministic True --tau 1 --target_update_interval 1000
Results
My results on Humanoid-v2 environment using SAC, SAC(hard update) and SAC(deterministic, hard update). This is a plot of average rewards at every 10000 step interval
Parameters
| Parameters | Value |
|---|---|
| Shared | - |
| optimizer | Adam |
learning rate(--lr) |
3x10−4 |
discount(--gamma) (γ) |
0.99 |
replay buffer size(--replay_size) |
1x106 |
| number of hidden layers (all networks) | 2 |
number of hidden units per layer(--hidden_size) |
256 |
number of samples per minibatch(--batch_size) |
256 |
| nonlinearity | ReLU |
| SAC | - |
target smoothing coefficient(--tau) (τ) |
0.005 |
target update interval(--target_update_interval) |
1 |
gradient steps(--updates_per_step) |
1 |
| SAC (Hard Update) | - |
target smoothing coefficient(--tau) (τ) |
1 |
target update interval(--target_update_interval) |
1000 |
gradient steps (except humanoids)(--updates_per_step) |
4 |
gradient steps (humanoids)(--updates_per_step) |
1 |
| Environment | Reward Scale |
|---|---|
| HalfCheetah-v2 | 5 |
| Hopper-v2 | 5 |
| Walker2d-v2 | 5 |
| Ant-v2 | 5 |
| Humanoid-v2 | 20 |