From c0936e148b84a2efc11cffc5726790c2e9d9e389 Mon Sep 17 00:00:00 2001 From: Shangtong Zhang Date: Sat, 3 Jun 2017 09:45:53 -0600 Subject: [PATCH] Update Readme --- README.md | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 5fa36da..3221d37 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,6 @@ # DeepRL -> Highly modularized implementation of popular deep RL algorithms powered by PyTorch +Highly modularized implementation of popular deep RL algorithms by PyTorch. My principal here is to +reuse as much components as I can through different algorithms and use as less tricks as I can. * Deep Q-Learning * Asynchronous One-Step Q-Learning * Asynchronous One-Step Sarsa @@ -8,8 +9,24 @@ >Tested with both classical control tasks (CartPole) and Atari games. +# Curves + +## Asynchronous Advantage Actor Critic (A3C) + +![alt text](DeepRL/images/A3C-PongNoFrameskip-v3.png) + +The network I used here is same as the network in DQN except the activation function +is **Elu** rather than Relu. The optimizer is **Adam** with non-shared parameters. +To my best knowledge, this network architecture is not the most suitable for A3C. +If you use a 42 * 42 input, add a LSTM layer, you will get much much much better training speed +than this. [GAE](http://www.breloff.com/DeepRL-OnlineGAE/) can also improve performance. +Another important thing is I didn't use lock for syncing up networks. Although I think there +should be a lock, locking can hurt the performance heavily (about 50%). + +The first 15 million frames took about 5 hours (16 processes) in a server with two Xeon E5-2620 v3. + # Usage -Detailed usage can be found in ```main.py``` +Detailed usage and all training details can be found in ```main.py``` # References * [Human Level Control through Deep Reinforcement Learning](https://www.nature.com/nature/journal/v518/n7540/full/nature14236.html)