diff --git a/README.md b/README.md index 458542b..7c2940e 100644 --- a/README.md +++ b/README.md @@ -15,6 +15,8 @@ Implemented algorithms: * Distributed Deep Deterministic Policy Gradient (Distributed DDPG, aka D3PG) * Parallelized Proximal Policy Optimization (P3O, similar to DPPO) * Action Conditional Video Prediction +* Categorical DQN (C51, Distributional DQN) +* N-Step DQN (similar to A2C) # Curves > Curves for CartPole are trivial so I didn't place it here. There isn't any fixed random seed. @@ -87,6 +89,11 @@ I use 8 threads and a two tanh hidden layer network, each hidden layer has 64 hi Prediction is sampled after 110K iterations and I only implemented one-step training +## Categorical DQN + +![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/CategoricalDQN.png) +A deterministic test episode is triggered every 10 episodes. 2.5M steps and 14 hours in total. + # Dependency > Tested in macOS 10.12 and CentO/S 6.8 * Open AI gym @@ -116,3 +123,4 @@ Prediction is sampled after 110K iterations and I only implemented one-step trai * [Proximal Policy Optimization Algorithms](https://arxiv.org/abs/1707.06347) * [Emergence of Locomotion Behaviours in Rich Environments](https://arxiv.org/abs/1707.02286) * [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://arxiv.org/abs/1507.08750) +* [A Distributional Perspective on Reinforcement Learning](https://arxiv.org/abs/1707.06887) \ No newline at end of file diff --git a/images/CategoricalDQN.png b/images/CategoricalDQN.png new file mode 100644 index 0000000..3457d5f Binary files /dev/null and b/images/CategoricalDQN.png differ diff --git a/main.py b/main.py index 13be964..cc5b966 100644 --- a/main.py +++ b/main.py @@ -402,7 +402,7 @@ def n_step_dqn_pixel_atari(name): config.num_workers = 8 config.task_fn = lambda: ParallelizedTask(task_fn, config.num_workers) config.optimizer_fn = lambda params: torch.optim.RMSprop(params, lr=0.00025, alpha=0.95, eps=0.01) - config.network_fn = lambda: NatureConvNet(config.history_length, task.action_dim, gpu=0) + config.network_fn = lambda: NatureConvNet(config.history_length, task.action_dim, gpu=1) config.policy_fn = lambda: GreedyPolicy(epsilon=1.0, final_step=1000000, min_epsilon=0.1) config.reward_shift_fn = lambda r: np.sign(r) config.discount = 0.99