Update README

README.md

@@ -15,6 +15,8 @@ Implemented algorithms:
 * Distributed Deep Deterministic Policy Gradient (Distributed DDPG, aka D3PG)
 * Parallelized Proximal Policy Optimization (P3O, similar to DPPO)
 * Action Conditional Video Prediction
+* Categorical DQN (C51, Distributional DQN)
+* N-Step DQN (similar to A2C)
 
 # Curves
 > Curves for CartPole are trivial so I didn't place it here. There isn't any fixed random seed.
@@ -87,6 +89,11 @@ I use 8 threads and a two tanh hidden layer network, each hidden layer has 64 hi
 
 Prediction is sampled after 110K iterations and I only implemented one-step training
 
+## Categorical DQN
+
+![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/CategoricalDQN.png)
+A deterministic test episode is triggered every 10 episodes. 2.5M steps and 14 hours in total.
+
 # Dependency
 > Tested in macOS 10.12 and CentO/S 6.8
 * Open AI gym
@@ -116,3 +123,4 @@ Prediction is sampled after 110K iterations and I only implemented one-step trai
 * [Proximal Policy Optimization Algorithms](https://arxiv.org/abs/1707.06347)
 * [Emergence of Locomotion Behaviours in Rich Environments](https://arxiv.org/abs/1707.02286)
 * [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://arxiv.org/abs/1507.08750)
+* [A Distributional Perspective on Reinforcement Learning](https://arxiv.org/abs/1707.06887)

main.py

@@ -402,7 +402,7 @@ def n_step_dqn_pixel_atari(name):
     config.num_workers = 8
     config.task_fn = lambda: ParallelizedTask(task_fn, config.num_workers)
     config.optimizer_fn = lambda params: torch.optim.RMSprop(params, lr=0.00025, alpha=0.95, eps=0.01)
-    config.network_fn = lambda: NatureConvNet(config.history_length, task.action_dim, gpu=0)
+    config.network_fn = lambda: NatureConvNet(config.history_length, task.action_dim, gpu=1)
     config.policy_fn = lambda: GreedyPolicy(epsilon=1.0, final_step=1000000, min_epsilon=0.1)
     config.reward_shift_fn = lambda r: np.sign(r)
     config.discount = 0.99