Add pybullet support

README.md

@@ -26,8 +26,16 @@ It's extremely difficult to reproduce results for Reinforcement Learning methods
 * Improve performance of KFAC, see kfac.py for more information
 * Run evaluation for all games and algorithms
 
+## Requirements
+
+* [PyTorch](http://pytorch.org/)
+* [Visdom](https://github.com/facebookresearch/visdom)
+* [OpenAI baselines](https://github.com/openai/baselines)
+
 ## Usage
 
+Start a `Visdom` server with `python -m visdom.server`, it will serve `http://localhost:8097/` by default.
+
 ### Atari
 #### A2C
 

envs.py

@@ -9,6 +9,10 @@ from baselines.common.atari_wrappers import wrap_deepmind
 
 def make_env(env_id, seed, rank, log_dir):
     def _thunk():
+        if env_id.find('Bullet') > -1:
+            import pybullet_envs
+            env = pybullet_envs.make(env_id)
+        else:
             env = gym.make(env_id)
         env.seed(seed + rank)
         env = bench.Monitor(env,