diff --git a/doc/source/rllib.rst b/doc/source/rllib.rst
index a9692bba4..e48a256ac 100644
--- a/doc/source/rllib.rst
+++ b/doc/source/rllib.rst
@@ -38,6 +38,18 @@ on a single node.
 These algorithms can be run on any `OpenAI Gym MDP <https://github.com/openai/gym>`__,
 including custom ones written and registered by the user.
 
+Installation
+------------
+
+RLlib has extra dependencies on top of **ray**:
+
+.. code-block:: bash
+
+  pip install tensorflow pyyaml gym[atari] opencv-python scipy
+
+For usage of PyTorch models, visit the `PyTorch website <http://pytorch.org/>`__
+for instructions on installing PyTorch.
+
 Getting Started
 ---------------
 
diff --git a/doc/source/tune.rst b/doc/source/tune.rst
index 1d4253316..c4d8769a3 100644
--- a/doc/source/tune.rst
+++ b/doc/source/tune.rst
@@ -14,7 +14,7 @@ This document describes Ray.tune, a hyperparameter tuning tool for long-running
 You can find the code for Ray.tune `here on GitHub <https://github.com/ray-project/ray/tree/master/python/ray/tune>`__.
 
 Getting Started
---------------- 
+---------------
 
 ::
 
diff --git a/python/ray/rllib/README.rst b/python/ray/rllib/README.rst
index 240438d04..7251af534 100644
--- a/python/ray/rllib/README.rst
+++ b/python/ray/rllib/README.rst
@@ -1,5 +1,5 @@
-RLlib: A Scalable Reinforcement Learning Library
-================================================
+Ray RLlib: A Composable and Scalable Reinforcement Learning Library
+===================================================================
 
 This README provides a brief technical overview of RLlib. See also the `user documentation <http://ray.readthedocs.io/en/latest/rllib.html>`__.