diff --git a/doc/source/index.rst b/doc/source/index.rst
index 9ce466a35..a3aefa20f 100644
--- a/doc/source/index.rst
+++ b/doc/source/index.rst
@@ -1,42 +1,18 @@
-Ray
-===
+What is Ray?
+============
-.. raw:: html
+.. include:: ray-overview/basics.rst
-
+Getting Started with Ray
+------------------------
-.. image:: https://github.com/ray-project/ray/raw/master/doc/source/images/ray_header_logo.png
+Check out :ref:`gentle-intro` to learn more about Ray and its ecosystem of libraries that enable things like distributed hyperparameter tuning,
+reinforcement learning, and distributed training.
-**Ray is a fast and simple framework for building and running distributed applications.**
-
-
-Ray is packaged with the following libraries for accelerating machine learning workloads:
-
-- `Tune`_: Scalable Hyperparameter Tuning
-- `RLlib`_: Scalable Reinforcement Learning
-- `RaySGD`_: Distributed Training Wrappers
-- :ref:`rayserve`
-
-
-Star us on `on GitHub`_. You can also get started by visiting our `Tutorials `_. For the latest wheels (nightlies), see the `installation page `__.
-
-.. _`on GitHub`: https://github.com/ray-project/ray
-.. _`RaySGD`: raysgd/raysgd.html
-
-.. tip:: Join our `community slack `_ to discuss Ray!
-
-
-Quick Start
------------
-
-First, install Ray with: ``pip install ray``
+Ray uses Tasks (functions) and Actors (Classes) to allow you to parallelize your Python code:
.. code-block:: python
- # Execute Python functions in parallel.
-
import ray
ray.init()
@@ -45,14 +21,7 @@ First, install Ray with: ``pip install ray``
return x * x
futures = [f.remote(i) for i in range(4)]
- print(ray.get(futures))
-
-To use Ray's actor model:
-
-.. code-block:: python
-
- import ray
- ray.init()
+ print(ray.get(futures)) # [0, 1, 4, 9]
@ray.remote
class Counter(object):
@@ -68,97 +37,29 @@ To use Ray's actor model:
counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
- print(ray.get(futures))
+ print(ray.get(futures)) # [1, 1, 1, 1]
-Visit the `Walkthrough `_ page a more comprehensive overview of Ray features.
-
-Ray programs can run on a single machine, and can also seamlessly scale to large clusters. To execute the above Ray script in the cloud, just download `this configuration file `__, and run:
-
-``ray submit [CLUSTER.YAML] example.py --start``
-
-Read more about `launching clusters `_.
-
-Tune Quick Start
-----------------
-
-`Tune`_ is a library for hyperparameter tuning at any scale. With Tune, you can launch a multi-node distributed hyperparameter sweep in less than 10 lines of code. Tune supports any deep learning framework, including PyTorch, TensorFlow, and Keras.
-
-.. note::
-
- To run this example, you will need to install the following:
-
- .. code-block:: bash
-
- $ pip install ray torch torchvision filelock
-
-
-This example runs a small grid search to train a CNN using PyTorch and Tune.
-
-.. literalinclude:: ../../python/ray/tune/tests/example.py
- :language: python
- :start-after: __quick_start_begin__
- :end-before: __quick_start_end__
-
-If TensorBoard is installed, automatically visualize all trial results:
-
-.. code-block:: bash
-
- tensorboard --logdir ~/ray_results
-
-.. _`Tune`: tune.html
-
-RLlib Quick Start
+The Ray Community
-----------------
-`RLlib`_ is an open-source library for reinforcement learning built on top of Ray that offers both high scalability and a unified API for a variety of applications.
+Ray is more than a framework for distributed applications but also an active community of developers,
+researchers, and folks that love machine learning.
-.. code-block:: bash
+You can join (and Star!) us on `on GitHub`_.
+You can also join `community slack `_ to discuss Ray! The community is extremely active in helping people succeed in building their ray applications.
- pip install tensorflow # or tensorflow-gpu
- pip install ray[rllib] # also recommended: ray[debug]
+You can also get started by visiting our `Tutorials `_. For the latest wheels (nightlies), see the `installation page `__.
-.. code-block:: python
-
- import gym
- from gym.spaces import Discrete, Box
- from ray import tune
-
- class SimpleCorridor(gym.Env):
- def __init__(self, config):
- self.end_pos = config["corridor_length"]
- self.cur_pos = 0
- self.action_space = Discrete(2)
- self.observation_space = Box(0.0, self.end_pos, shape=(1, ))
-
- def reset(self):
- self.cur_pos = 0
- return [self.cur_pos]
-
- def step(self, action):
- if action == 0 and self.cur_pos > 0:
- self.cur_pos -= 1
- elif action == 1:
- self.cur_pos += 1
- done = self.cur_pos >= self.end_pos
- return [self.cur_pos], 1 if done else 0, done, {}
-
- tune.run(
- "PPO",
- config={
- "env": SimpleCorridor,
- "num_workers": 4,
- "env_config": {"corridor_length": 5}})
-
-.. _`RLlib`: rllib.html
+.. _`on GitHub`: https://github.com/ray-project/ray
More Information
-----------------
+================
Here are some talks, papers, and press coverage involving Ray and its libraries. Please raise an issue if any of the below links are broken!
Blog and Press
-~~~~~~~~~~~~~~
+--------------
- `Modern Parallel and Distributed Python: A Quick Tutorial on Ray `_
- `Why Every Python Developer Will Love Ray `_
@@ -180,7 +81,7 @@ Blog and Press
.. _`Ray Blog`: https://ray-project.github.io/
Talks (Videos)
-~~~~~~~~~~~~~~
+--------------
- `Programming at any Scale with Ray | SF Python Meetup Sept 2019 `_
- `Ray for Reinforcement Learning | Data Council 2019 `_
@@ -192,13 +93,14 @@ Talks (Videos)
- `Tune: Distributed Hyperparameter Search | RISECamp 2018 `_
Slides
-~~~~~~
+------
+
- `Talk given at UC Berkeley DS100 `_
- `Talk given in October 2019 `_
- [Tune] `Talk given at RISECamp 2019 `_
Academic Papers
-~~~~~~~~~~~~~~~
+---------------
- `Ray paper`_
- `Ray HotOS paper`_
@@ -211,7 +113,7 @@ Academic Papers
.. _`Tune paper`: https://arxiv.org/abs/1807.05118
Getting Involved
-----------------
+================
- `ray-dev@googlegroups.com`_: For discussions about development or any general
questions.
@@ -226,10 +128,13 @@ Getting Involved
+
+
.. toctree::
:maxdepth: -1
- :caption: Installation
+ :caption: Overview of Ray
+ ray-overview/index.rst
installation.rst
.. toctree::
diff --git a/doc/source/ray-overview/basics.rst b/doc/source/ray-overview/basics.rst
new file mode 100644
index 000000000..514581267
--- /dev/null
+++ b/doc/source/ray-overview/basics.rst
@@ -0,0 +1,31 @@
+
+.. raw:: html
+
+
+
+.. image:: https://github.com/ray-project/ray/raw/master/doc/source/images/ray_header_logo.png
+
+**Ray is a fast and simple framework for building and running distributed applications.**
+
+Ray accomplishes this mission by:
+
+1. Providing simple primitives for building and running distributed applications.
+2. Enabling end users to parallelize single machine code, with little to zero code changes.
+3. Including a large ecosystem of applications, libraries, and tools on top of the core Ray to enable complex applications.
+
+**Ray Core** provides the simple primitives for application building.
+
+On top of **Ray Core** are several libraries for solving problems in machine learning:
+
+- :ref:`tune-index`
+- :ref:`rllib-index`
+- :ref:`sgd-index`
+- :ref:`rayserve`
+
+Ray also has a number of other community contributed libraries:
+
+- :doc:`../pandas_on_ray`
+- :doc:`../joblib`
+- :doc:`../multiprocessing`
\ No newline at end of file
diff --git a/doc/source/ray-overview/index.rst b/doc/source/ray-overview/index.rst
new file mode 100644
index 000000000..aade3f8fc
--- /dev/null
+++ b/doc/source/ray-overview/index.rst
@@ -0,0 +1,160 @@
+.. _gentle-intro:
+
+============================
+A Gentle Introduction to Ray
+============================
+
+.. include:: basics.rst
+
+This tutorial will provide a tour of the core features of Ray.
+
+First, install Ray with: ``pip install ray``, and now we can execute some Python in parallel.
+
+Parallelizing Python Functions with Ray Tasks
+=============================================
+
+First, import ray and ``init`` the Ray service.
+Then decorate your function with ``@ray.remote`` to declare that you want to run this function
+remotely. Lastly, call that function with ``.remote()`` instead of calling it normally. This remote call yields a future, or ``ObjectID`` that you can then
+fetch with ``ray.get``.
+
+.. code-block:: python
+
+ import ray
+ ray.init()
+
+ @ray.remote
+ def f(x):
+ return x * x
+
+ futures = [f.remote(i) for i in range(4)]
+ print(ray.get(futures)) # [0, 1, 4, 9]
+
+In the above code block we defined some Ray Tasks. While these are great for stateless operations, sometimes you
+must maintain the state of your application. You can do that with Ray Actors.
+
+Parallelizing Python Classes with Ray Actors
+==============================================
+
+Ray provides actors to allow you to parallelize an instance of a class in Python.
+When you instantiate a class that is a Ray actor, Ray will start a remote instance
+of that class in the cluster. This actor can then execute remote method calls and
+maintain its own internal state.
+
+.. code-block:: python
+
+ import ray
+ ray.init() # Only call this once.
+
+ @ray.remote
+ class Counter(object):
+ def __init__(self):
+ self.n = 0
+
+ def increment(self):
+ self.n += 1
+
+ def read(self):
+ return self.n
+
+ counters = [Counter.remote() for i in range(4)]
+ [c.increment.remote() for c in counters]
+ futures = [c.read.remote() for c in counters]
+ print(ray.get(futures)) # [1, 1, 1, 1]
+
+
+An Overview of the Ray Libraries
+================================
+
+Ray has a rich ecosystem of libraries and frameworks built on top of it. The main ones being:
+
+- :ref:`tune-index`
+- :ref:`rllib-index`
+- :ref:`sgd-index`
+- :ref:`rayserve`
+
+
+Tune Quick Start
+----------------
+
+`Tune`_ is a library for hyperparameter tuning at any scale. With Tune, you can launch a multi-node distributed hyperparameter sweep in less than 10 lines of code. Tune supports any deep learning framework, including PyTorch, TensorFlow, and Keras.
+
+.. note::
+
+ To run this example, you will need to install the following:
+
+ .. code-block:: bash
+
+ $ pip install ray torch torchvision filelock
+
+
+This example runs a small grid search to train a CNN using PyTorch and Tune.
+
+.. literalinclude:: ../../../python/ray/tune/tests/example.py
+ :language: python
+ :start-after: __quick_start_begin__
+ :end-before: __quick_start_end__
+
+If TensorBoard is installed, automatically visualize all trial results:
+
+.. code-block:: bash
+
+ tensorboard --logdir ~/ray_results
+
+.. _`Tune`: tune.html
+
+RLlib Quick Start
+-----------------
+
+`RLlib`_ is an open-source library for reinforcement learning built on top of Ray that offers both high scalability and a unified API for a variety of applications.
+
+.. code-block:: bash
+
+ pip install tensorflow # or tensorflow-gpu
+ pip install ray[rllib] # also recommended: ray[debug]
+
+.. code-block:: python
+
+ import gym
+ from gym.spaces import Discrete, Box
+ from ray import tune
+
+ class SimpleCorridor(gym.Env):
+ def __init__(self, config):
+ self.end_pos = config["corridor_length"]
+ self.cur_pos = 0
+ self.action_space = Discrete(2)
+ self.observation_space = Box(0.0, self.end_pos, shape=(1, ))
+
+ def reset(self):
+ self.cur_pos = 0
+ return [self.cur_pos]
+
+ def step(self, action):
+ if action == 0 and self.cur_pos > 0:
+ self.cur_pos -= 1
+ elif action == 1:
+ self.cur_pos += 1
+ done = self.cur_pos >= self.end_pos
+ return [self.cur_pos], 1 if done else 0, done, {}
+
+ tune.run(
+ "PPO",
+ config={
+ "env": SimpleCorridor,
+ "num_workers": 4,
+ "env_config": {"corridor_length": 5}})
+
+.. _`RLlib`: rllib.html
+
+Where to go next?
+=================
+
+
+Visit the `Walkthrough `_ page a more comprehensive overview of Ray features.
+
+Ray programs can run on a single machine, and can also seamlessly scale to large clusters. To execute the above Ray script in the cloud, just download `this configuration file `__, and run:
+
+``ray submit [CLUSTER.YAML] example.py --start``
+
+Read more about `launching clusters `_.
diff --git a/doc/source/raysgd/raysgd.rst b/doc/source/raysgd/raysgd.rst
index a20a566ba..e7b8fdf50 100644
--- a/doc/source/raysgd/raysgd.rst
+++ b/doc/source/raysgd/raysgd.rst
@@ -1,3 +1,6 @@
+.. _sgd-index:
+
+=====================================
RaySGD: Distributed Training Wrappers
=====================================
diff --git a/doc/source/rllib.rst b/doc/source/rllib.rst
index 035e2477e..276794b0f 100644
--- a/doc/source/rllib.rst
+++ b/doc/source/rllib.rst
@@ -1,3 +1,5 @@
+.. _rllib-index:
+
RLlib: Scalable Reinforcement Learning
======================================