wassname/DeepRL
1
0
Fork 0
mirror of https://github.com/wassname/DeepRL.git synced 2026-08-20 12:00:17 +08:00
Code Issues Packages Projects Releases Wiki Activity

Update README

Browse Source
This commit is contained in:
Shangtong Zhang
2017-11-11 21:05:03 -07:00
parent 0fb70fa484
commit d47bb7a59a
1 changed files with 5 additions and 5 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+5 -5
View File
@@ -28,6 +28,8 @@ Xeon E5-2620 v3 and Titan X. For Breakout, test is triggered every 1000 episodes
In total, 16M frames cost about 4 days and 10 hours. For Pong, test is triggered
every 10 episodes with no repetition. In total, 4M frames cost about 18 hours.
I referred this [repo](https://github.com/transedward/pytorch-dqn).
## Discrete A3C
![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/A3C-Pong.png)
@@ -40,6 +42,8 @@ Training of A3C took about 2 hours (16 processes) in a server with two Xeon E5-2
Those value based async methods do work but I don't know how to make them stable.
This is the test curve. Test is triggered in a separate deterministic test process every 50K frames.
I referred this [repo](https://github.com/ikostrikov/pytorch-a3c) for the parallelization.
## Continuous A3C
![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/Continuous-A3C.png)
@@ -68,7 +72,7 @@ The difference between my implementation and [DeepMind's DPPO](https://arxiv.org
1. PPO stands for different algorithms.
2. I use a much simpler A3C-like synchronization protocol.
The body of PPO is based on [this](https://github.com/alexis-jacq/Pytorch-DPPO).
The body of PPO is based on this [repo](https://github.com/alexis-jacq/Pytorch-DPPO).
However that implementation has two critical bugs at least at this [commit](https://github.com/ghliu/pytorch-ddpg/tree/ffea335ee53f2ff90b6d7eaf9d0cee705270c0f1).
Its computation of the clipped loss is correct with one-dimensional action by accident,
but is wrong with high-dimensional action. And its computation of entropy is wrong in any case.
@@ -106,7 +110,3 @@ mkdir data log
* [Trust Region Policy Optimization](https://arxiv.org/abs/1502.05477)
* [Proximal Policy Optimization Algorithms](https://arxiv.org/abs/1707.06347)
* [Emergence of Locomotion Behaviours in Rich Environments](https://arxiv.org/abs/1707.02286)
* [transedward/pytorch-dqn](https://github.com/transedward/pytorch-dqn)
* [ikostrikov/pytorch-a3c](https://github.com/ikostrikov/pytorch-a3c)
* [ghliu/pytorch-ddpg](https://github.com/ghliu/pytorch-ddpg)
* [alexis-jacq/Pytorch-DPPO](https://github.com/alexis-jacq/Pytorch-DPPO)