Update README

README.md

@@ -23,31 +23,24 @@ Following asynchronous algorithms are removed now in this repo but can be found
 # Curves
 > Curves for CartPole are trivial so I didn't place it here. And there isn't any fixed random seed. The curves are generated in the same manner as OpenAI baselines (one run and smoothed by recent 100 episodes)
 ## DQN
-> BreakoutNoFrameSkip-v4
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/dqn_pixel_atari-180407-01414.png)
 
 ## Categorical DQN
-> BreakoutNoFrameskip-v4
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/categorical_dqn_pixel_atari-180407-094006.png)
 
 ## Quantile Regression DQN
-> BreakoutNoFrameskip-v4
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/quantile_regression_dqn_pixel_atari-180407-01604.png)
 
 ## A2C 
-> BreakoutNoFrameskip-v4
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/a2c_pixel_atari-180407-92711.png)
 
 ## N-Step Q-Learning
-> BreakoutNoFrameskip-v4
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/dqn_pixel_atari-180407-01414.png)
 
 ## DDPG 
-> RoboschoolHopper-v1
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/ddpg_continuous-180407-234141.png)
 
 ## PPO 
-> RoboschoolHopper-v1
 ![Loading...](https://raw.githubusercontent.com/ShangtongZhang/DeepRL/master/images/ppo_continuous-180408-002056.png)
 
 ## Action Conditional Video Prediction
@@ -89,5 +82,5 @@ Prediction is sampled after 110K iterations, and I only implemented one-step tra
 * [Action-Conditional Video Prediction using Deep Networks in Atari Games](https://arxiv.org/abs/1507.08750)
 * [A Distributional Perspective on Reinforcement Learning](https://arxiv.org/abs/1707.06887)
 * [Distributional Reinforcement Learning with Quantile Regression](https://arxiv.org/abs/1710.10044)
-* [DeepMind Control Suite](https://arxiv.org/abs/1801.00690)
-* [OpenAI Baselines](https://github.com/openai/baselines), [Ilya Kostrikov](https://github.com/ikostrikov/pytorch-a2c-ppo-acktr)
+
+* Some hyper-parameters are from [DeepMind Control Suite](https://arxiv.org/abs/1801.00690), [OpenAI Baselines](https://github.com/openai/baselines) and [Ilya Kostrikov](https://github.com/ikostrikov/pytorch-a2c-ppo-acktr)

main.py

@@ -280,11 +280,12 @@ def plot():
     # name = 'log/ppo_continuous-180408-002056'
     # plotter.plot_results([name])
     # plt.show()
-    names = ['a2c_pixel_atari-180407-92711',
-            'categorical_dqn_pixel_atari-180407-094006',
-            'dqn_pixel_atari-180407-01414',
-            'quantile_regression_dqn_pixel_atari-180407-01604',
-             'n_step_dqn_pixel_atari-180408-001104',
+    names = [
+            # 'a2c_pixel_atari-180407-92711',
+            # 'categorical_dqn_pixel_atari-180407-094006',
+            # 'dqn_pixel_atari-180407-01414',
+            # 'quantile_regression_dqn_pixel_atari-180407-01604',
+            #  'n_step_dqn_pixel_atari-180408-001104',
              'ppo_continuous-180408-002056',
              'ddpg_continuous-180407-234141'
              ]

utils/plot.py

@@ -47,7 +47,7 @@ class Plotter:
         xy_list = [[x, y, self.window_func(x, y, episode_window, np.mean)] for x, y in xy_list]
         return xy_list
 
-    def plot_results(self, dirs, max_timesteps=1e8, x_axis=X_TIMESTEPS, episode_window=100):
+    def plot_results(self, dirs, max_timesteps=1e8, x_axis=X_TIMESTEPS, episode_window=100, title=None):
         import matplotlib.pyplot as plt
         plt.ticklabel_format(axis='x', style='sci', scilimits=(1, 1))
         xy_list = self.load_results(dirs, max_timesteps, x_axis, episode_window)
@@ -56,3 +56,5 @@ class Plotter:
             plt.plot(smoothed[0], smoothed[1], color=color)
         plt.xlabel(x_axis)
         plt.ylabel("Episode Rewards")
+        if title is not None:
+            plt.title(title)