From 5873831c21f0c2441149db5db8148dfe5eaaf1b8 Mon Sep 17 00:00:00 2001
From: Robert Nishihara <robertnishihara@gmail.com>
Date: Wed, 6 Jul 2016 13:51:32 -0700
Subject: [PATCH] basic tutorials (#204)

---
 README.md                     |  81 +------------
 doc/about-the-system.md       |  32 ++++++
 doc/basic-usage.md            | 207 ++++++++++++++++++++++++++++++++++
 doc/download-and-setup.md     |  53 +++++++++
 doc/figures/compgraph1.png    | Bin 0 -> 2819 bytes
 doc/figures/compgraph2.png    | Bin 0 -> 4557 bytes
 doc/figures/compgraph3.png    | Bin 0 -> 2645 bytes
 doc/introduction.md           |  49 ++++++++
 doc/tutorial.md               | 118 +++++++++++++++++++
 doc/using-ray-on-a-cluster.md |  79 +++++++++++++
 scripts/example_functions.py  |   4 +
 11 files changed, 543 insertions(+), 80 deletions(-)
 create mode 100644 doc/about-the-system.md
 create mode 100644 doc/basic-usage.md
 create mode 100644 doc/download-and-setup.md
 create mode 100644 doc/figures/compgraph1.png
 create mode 100644 doc/figures/compgraph2.png
 create mode 100644 doc/figures/compgraph3.png
 create mode 100644 doc/introduction.md
 create mode 100644 doc/tutorial.md
 create mode 100644 doc/using-ray-on-a-cluster.md

diff --git a/README.md b/README.md
index a0c64081e..7302c53ae 100644
--- a/README.md
+++ b/README.md
@@ -5,83 +5,4 @@
 Ray is an experimental distributed execution framework with a Python-like
 programming model. It is under development and not ready for general use.
 
-## Example Code
-
-### Loading ImageNet
-TODO: fill this out.
-
-## Design Decisions
-
-For a description of our design decisions, see
-
-- [Reference Counting](doc/reference-counting.md)
-- [Aliasing](doc/aliasing.md)
-- [Scheduler](doc/scheduler.md)
-
-## Setup
-
-### Linux, Mac, and other Unix-based systems
-
-After running these instruction, add the line `source "$RAY_ROOT/setup-env.sh"` in your `~/.bashrc` file manually, where "$RAY_ROOT" is the path of the directory containing `setup-env.sh`.
-
-1. sudo apt-get update
-2. sudo apt-get install git
-3. git clone https://github.com/amplab/ray.git
-4. cd ray
-5. ./setup.sh
-6. ./build.sh
-7. source setup-env.sh
-
-### Windows
-
-**Note:** A batch file is provided that clones any missing third-party libraries and applies patches to them.
-Do not attempt to open the solution before the batch file applies the patches; otherwise, if the projects have been modified, the patches may be rejected, and you may be forced to revert your changes before re-running the batch file.
-
-1. Install Microsoft Visual Studio 2015
-2. Install Git
-3. git clone https://github.com/amplab/ray.git
-4. ray\thirdparty\download_thirdparty.bat
-
-## Installing Ray on a cluster
-
-These instructions work on EC2, but they may require some modifications to run
-on your own cluster. In particular, on EC2, running `sudo` does not require a
-password, and we currently don't handle the case where a password is needed.
-
-1. Create a file `nodes.txt` of the IP addresses of the nodes in the cluster.
-For example
-
-        52.50.28.103
-        52.51.210.207
-2. Make sure that the nodes can all communicate with one another. On EC2, this
-can be done by creating a new security group and adding the inbound rule "all
-traffic" and adding the outbound rule "all traffic". Then add all of the nodes
-in your cluster to that security group.
-
-3. Run something like
-    ```
-    python scripts/cluster.py --nodes nodes.txt \
-                              --key-file key.pem \
-                              --username ubuntu \
-                              --installation-directory /home/ubuntu/
-    ```
-where you replace `nodes.txt`, `key.pem`, `ubuntu`, and `/home/ubuntu/` by the
-appropriate values. This assumes that you can connect to each IP address in
-`nodes.txt` with the command
-    ```
-    ssh -i key.pem ubuntu@<ip-address>
-    ```
-4. The previous command should open a Python interpreter. To install Ray on the
-cluster, run `install_ray()` in the interpreter. The interpreter should block
-until the installation has completed.
-5. To check that the installation succeeded, you can ssh to each node, cd into
-the directory `ray/test/`, and run the tests (e.g., `python runtest.py`).
-6. Now that Ray has been installed, you can start the cluster (the scheduler,
-object stores, and workers) with the command
-`start_ray("/home/ubuntu/ray/scripts/default_worker.py")`, where the argument is
-the path on each node in the cluster to the worker code that you would like to
-use. The workers can be restarted with
-`restart_workers("/home/ubuntu/ray/scripts/default_worker.py")`, for example if
-you wish to update the application code running on the workers. The cluster
-processes (the scheduler, the object stores, and the workers) can be stopped
-with `stop_ray()`.
+Read this [introduction to Ray](doc/introduction.md).
diff --git a/doc/about-the-system.md b/doc/about-the-system.md
new file mode 100644
index 000000000..52b7eb164
--- /dev/null
+++ b/doc/about-the-system.md
@@ -0,0 +1,32 @@
+## About the System
+
+This document describes the current architecture of Ray. However, some of these
+decisions are likely to change.
+
+### Components
+
+A Ray cluster consists of several components.
+
+- One scheduler
+- Multiple workers per node
+- One object store per node
+- One (or more) drivers
+
+#### The scheduler
+
+The scheduler assigns tasks to the workers.
+
+#### The workers
+
+The workers execute tasks and submit tasks to the scheduler.
+
+#### The object store
+
+The object store shares objects between the worker processes on the same node so
+that the workers don't need to each have their own copies of the objects.
+
+#### The driver
+
+The driver submits tasks to the scheduler. If you use Ray in a script, the
+Python process running the script is the driver. If you use Ray interactively
+through a shell, the shell process is the driver.
diff --git a/doc/basic-usage.md b/doc/basic-usage.md
new file mode 100644
index 000000000..feeeb09e2
--- /dev/null
+++ b/doc/basic-usage.md
@@ -0,0 +1,207 @@
+## Basic Usage
+
+To use Ray, you need to understand the following:
+
+- How Ray uses object references to represent immutable remote objects.
+- How Ray constructs computation graphs using remote functions.
+
+### Immutable remote objects
+
+In Ray, we can create and manipulate objects. We refer to these objects as
+**remote objects**, and we use **object references** to refer to them. Remote
+objects are stored in **object stores**, and there is one object store per node
+in the cluster. In the cluster setting, we may not actually know which machine
+each object lives on.
+
+An **object reference** is essentially a unique ID that can be used to refer to
+a remote object. If you're familiar with Futures, our object references are
+conceptually similar.
+
+We assume that remote objects are immutable. That is, their values cannot be
+changed after creation. This allows remote objects to be replicated in multiple
+object stores without needing to synchronize the copies.
+
+#### Put and Get
+
+The commands `ray.get` and `ray.put` can be used to convert between Python
+objects and object references, as shown in the example below.
+```python
+>>> x = [1, 2, 3]
+>>> ray.put(x)
+<ray.ObjRef at 0x1031baef0>
+```
+
+The command `ray.put(x)` would be run by a worker process or by the driver
+process (the driver process is the one running your script). It takes a Python
+object and copies it to the local object store (here *local* means *on the same
+node*). Once the object has been stored in the object store, its value cannot be
+changed.
+
+In addition, `ray.put(x)` returns an object reference, which is essentially an
+ID that can be used to refer to the newly created remote object. If we save the
+object reference in a variable with `ref = ray.put(x)`, then we can pass `ref`
+into remote functions, and those remote functions will operate on the
+corresponding remote object.
+
+The command `ray.get(ref)` takes an object reference and creates a Python object
+from the corresponding remote object. For some objects like arrays, we can use
+shared memory and avoid copying the object. For other objects, this currently
+copies the object from the object store into the memory of the worker process.
+If the remote object corresponding to the object reference `ref` does not live
+on the same node as the worker that calls `ray.get(ref)`, then the remote object
+will first be copied from an object store that has it to the object store that
+needs it.
+```python
+>>> ref = ray.put([1, 2, 3])
+>>> ray.get(ref)
+[1, 2, 3]
+```
+
+If the remote object corresponding to the object reference `ref` has not been
+created yet, the command `ray.get(ref)` will wait until the remote object has
+been created.
+
+### Computation graphs in Ray
+
+Ray represents computation with a directed acyclic graph of tasks. Tasks are
+added to this graph by calling **remote functions**.
+
+For example, a normal Python function looks like this.
+```python
+def add(a, b):
+  return a + b
+```
+A remote function in Ray looks like this.
+```python
+@ray.remote([int, int], [int])
+def add(a, b):
+  return a + b
+```
+
+The information passed to the `@ray.remote` decorator includes type information
+for the arguments and for the return values of the function. Because of the
+distinction that we make between *submitting a task* and *executing the task*,
+we require type information so that we can catch type errors when the remote
+function is called instead of catching them when the task is actually executed.
+
+However, the only piece of information that is fundamentally required by the
+system is the number of return values (because the system must assign the
+correct number of object references to the outputs before the function has
+actually executed and produced any outputs).
+
+#### Remote functions
+
+Whereas in regular Python, calling `add(1, 2)` would return `3`, in Ray, calling
+`add(1, 2)` does not actually execute the task. Instead, it adds a task to the
+computation graph and immediately returns an object reference to the output of
+the computation.
+
+```python
+>>> ref = add(1, 2)
+>>> ray.get(ref)
+3
+```
+
+There is a sharp distinction between *submitting a task* and *executing the
+task*. When a remote function is called, the task of executing that function is
+submitted to the scheduler, and the scheduler immediately returns object
+references for the outputs of the task. However, the task will not be executed
+until the scheduler actually schedules the task on a worker.
+
+When a task is submitted, each argument may be passed in by value or by object
+reference. For example, these lines have the same behavior.
+
+```python
+>>> add(1, 2)
+>>> add(1, ray.put(2))
+>>> add(ray.put(1), ray.put(2))
+```
+
+Remote functions never return actual values, they always return object
+references.
+
+When the remote function is actually executed, it operates on Python objects.
+That is, if the remote function was called with any object references, the
+Python objects corresponding to those object references will be retrieved and
+passed into the actual execution of the remote function.
+
+#### Blocking computation
+
+In a regular Python script, the specification of a computation is intimately
+linked to the actual execution of the code. For example, consider the following
+code.
+```python
+result = []
+for i in range(10):
+  result.append(np.zeros(size=[100, 100]))
+```
+
+At the core of the above script, there are 10 separate tasks, each of which
+generates a 100x100 matrix of zeros. These tasks do not depend on each other, so
+in principle, they could be executed in parallel. However, in the above
+implementation, they will be executed serially.
+
+Ray gets around this by representing computation as a graph of tasks, where some
+tasks depend on the outputs of other tasks and where tasks can be executed once
+their dependencies have been executed.
+
+For example, suppose we define the remote function `zeros` to be a wrapper
+around `np.zeros`.
+```python
+from typing import List
+import numpy as np
+
+@ray.remote([List[int]], [np.ndarray])
+def zeros(shape):
+  return np.zeros(shape)
+```
+Then we can write
+```python
+result_refs = []
+for i in range(10):
+  result.append(zeros([100, 100]))
+```
+This adds 10 tasks to the computation graph, with no dependencies between the
+tasks.
+
+The computation graph looks like this.
+
+<p align="center">
+  <img src="figures/compgraph2.png">
+</p>
+
+In this figure, boxes are tasks and ovals are objects.
+
+The box that says `op-root` in it just refers to the overall script itself. The
+dotted lines indicate that the script launched 10 tasks (tasks are denoted by
+rectangular boxes). The solid lines indicate that each task produces one output
+(represented by an oval).
+
+It is clear from the computation graph that these ten tasks can be executed in
+parallel.
+
+Computation graphs encode dependencies. For example, suppose we define
+```python
+ray.remote([np.ndarray, np.ndarray], [np.ndarray])
+def dot(a, b):
+  return np.dot(a, b)
+```
+Then we run
+```python
+aref = zeros([10, 10])
+bref = zeros([10, 10])
+cref = dot(aref, bref)
+```
+The corresponding computation graph looks like this.
+
+<p align="center">
+  <img src="figures/compgraph3.png" width="300">
+</p>
+
+
+The three dashed lines indicate that the script launched three tasks (the two
+`zeros` tasks and the one `dot` task). Each task produces a single output, and
+the `dot` task depends on the outputs of the two `zeros` tasks.
+
+This makes it clear that the two `zeros` tasks can execute in parallel but that
+the `dot` task must wait until the two `zeros` tasks have finished.
diff --git a/doc/download-and-setup.md b/doc/download-and-setup.md
new file mode 100644
index 000000000..2090abadb
--- /dev/null
+++ b/doc/download-and-setup.md
@@ -0,0 +1,53 @@
+## Download and Setup
+
+Ray must currently be built from source.
+
+### Clone the Ray repository
+
+```
+git clone https://github.com/amplab/ray.git
+```
+These instructions will install the latest master branch for Ray.
+
+### Installation for Ubuntu and Mac OS X
+
+For convenience, we provide a setup script that pulls the necessary
+dependencies.
+
+```
+cd ray
+./setup.sh # This builds all necessary third party libraries (e.g., gRPC and Apache Arrow). It will require a sudo password.
+./build.sh # This builds Ray.
+source setup-env.sh # This adds Ray to your Python path.
+```
+
+For convenience, you may also want to add the line `source
+"$RAY_ROOT/setup-env.sh"` to your `~/.bashrc` file manually, where `$RAY_ROOT`
+is the Ray directory (e.g., `/home/ubuntu/ray`).
+
+To test if the installation was successful, try running some tests.
+
+### Installation for Windows
+
+Ray currently does not run on Windows. However, it can be compiled with the
+following instructions.
+
+**Note:** A batch file is provided that clones any missing third-party libraries
+and applies patches to them. Do not attempt to open the solution before the
+batch file applies the patches; otherwise, if the projects have been modified,
+the patches may be rejected, and you may be forced to revert your changes before
+re-running the batch file.
+
+1. Install Microsoft Visual Studio 2015
+2. Install Git
+3. `git clone https://github.com/amplab/ray.git`
+4. `ray\thirdparty\download_thirdparty.bat`
+
+### Test if the installation succeeded
+
+Try running some tests.
+
+```
+python test/runtest.py # This tests basic functionality.
+python test/array_test.py # This tests some array libraries.
+```
diff --git a/doc/figures/compgraph1.png b/doc/figures/compgraph1.png
new file mode 100644
index 0000000000000000000000000000000000000000..9abd43a362b2a9716c915d1e42ddde6daaf2f2ee
GIT binary patch
literal 2819
zcmZ{mdpHyNAIE1~o9o;)!%Qicj{D9p$zsYax1De(i-}HdEw{~`v15p&6iL~rlkhVn
zvHh$pPDrF&=E!A^+rmas)Lee+_uuc2^Ld`{=kt9&&*%9(@6YFXJ>Tb>hx2q+kVVJ>
z000G7ms4i|0ALCL09Ka*Niu806<v}5!MXc5?(OYKuD&ma(j`GA+U0B<03fIO<46JS
z-h%@GvJYHOVSN(Eg_9iO2Y(Idnp+T+myJS<Ic1|DclPt+5gpQB440mbkONV<Ves6$
znd?C8g5$F`Eajowc)f;pn~Nst0RbHI1?#%qrgF04Mbv4VYL^QKcQf1Dr`fg|S=4{9
zS@)_u$nlrlSOs)juA5pb?H&L;)s=BX3Zke~8g^<QU#kSmaa%(LduKW=M8?h!lrSC!
zRwfX4;yc$c(DOB}X`Z|_Og?nX1}3^Z&i}SPZB{T^$u=*nmAcX_PQJb0w<B$5%H9WF
z_q9KS{VT<!@JP+OecD-Ar~*60Tu=&45p!KXlAr8Zz^X5Dvse-6Kc9sjxf@K+>!K=a
zgO4ugA>t2Qd%k#+PnyA$uILcuR#~ATrrWDe+APGs7{7XS#nhpp*2A(wdc!>kF^aKi
zt+**~*~722=Yd}Z=-=+h2nPvrM!3&qwr>Myl~OCJ_LD~`ob-!>??FS}G^!rFz~F{8
zt_llH?+e*DsX2_?SOVQGNA5$DVRk{Z(mpfaUp7bfK0cf9&kV*w#jA9pkx<>*1P-J*
zhP<xJ4~VfyJajm{5&VGiFPF1H6J-MT=Y0v0(e9vmfl&A5w-~$Fzby-XL*sgHSnbX3
z=AWa!yCa=`X5H@JW=jcrfwDQUbM%Lex4$?7v3nl!4#Xs>^wXJ0QAsKPclp|5iLLuu
zN~ik`K~tPE>Ss}1L0!N5pnZ~E#6Vqt&W(t#CHy<m=jbEHXZ){!=Du??@A0s1u;h>I
z4g^0s{BoqBzNR@6^5He5K1<YRdRHaEiqHy7+?+z}%?9pS$`Xa#W&Om1uR1?75RRW0
z8X7Xv_rA|$?1q+rUto5tfrhIu{u0h)K<(0PpOdbfY2A()HZnfh4W8PY5+%0=Z#}*%
z)DKHb9r_7){xVq0Z7l$nY6#B$Bs(qhmi~roDEIunFNE{WWzS=diL=Lk8Us@s>ateN
z+KC}jCmwRshiV}w@X1H(lUi7Pl+2qW9j&WuGtRLs)?d3O-*8Wp!=UON@4{l23p>}|
zo~;%da0Tg0CD8k;({?ng#o6f$uSd_PU8iC<g5P7Af_hXJQMo5k=pSM}NQmZ?Dx?@%
zO`W}9zTQj~`aC!wJsqAfv}c4)YgTJxrO#sXmo90A4t8W|ASRT5(89$k%0a?#vs?{C
zyHly2Ulc?$71<bcrSU}t48+}!i>9O-A|+Wb^_5EOHONWym~)n&z%q(K|E!uRE05%d
ziPYrIIRiW!8IN`DEEB6N;Myu&C546W4Nt1e<vMey>|`BS1vq|&fYTk+hH0yjpz7>E
z=ivn1_jXL#qXIYb(R?Pi`peN8k&_8KGLi<`p^SZh(u8m@8M8h#3rx!+l6}jAh{uwP
zS?{0R5ECNriJe(J#AxC)vYku$a8HhbtO{qiHgSHD^KOBPHBR}C2Rg7EDZSz-)ees@
zo)x6X1NPZS>6?CslnMWdR<RGWF1jiZ29*^rwZzmt*@&un;oTrjOF2F*%b4P|_}=-s
ztKCh?gQum7g`yYQjJk8?w7`w!5tlz%>_Rxbe-gH<q%KlF8p|+)vH>gKjnHBA<BvVn
zq8NRRc@W`T_BiFyB?B|`<gu|0r`GiQ&O_XMeRx76C6#RXGZ-x(B==-dbFrtX+zrO~
zKFUTJDD_Fa;3Ga^bIR#2*X7z*s7GD(q1VC}%01?j*aKgozRna9X!&BPN)f*F@gHSP
zwefm<iuI*wZ{>O=Ai1aIuxcUT%7v)bvo`QS#6hz3O-%oh39FBXooy=x`5(x#pFie{
zMotqrs>dp+2a>{#A5JYj+clxD7vacdGFdH&CY{)HAA!?&_kzjQD(QPOazgIFRsXQ)
zA<#=Ej>_#XD;Tm<9=4}?pu>2`H<Pxo_(6tk>D4qmK~8~;V!ZLWYy+qCnTuIqj(o9S
z;x*>KbkJkAStajlFosdpUr{?f*GfUbeWAFP@OkXG^O9d!ku&D0mRno*g-fF*Ery+P
z=sO5kWkQ60w(S(()tD@4J4k3|Ue-)wG%>41t-j!Rm*<@DpHPiPt*Fq&<ulpK8Cjl_
zbxl&kw_1c66QZ<i@Zcog@9&~iI?z*rF9#G{8HWmnccMX(GR33N+%yHLR!xBgqh;TD
zwKlSS&KytEm%wNaM=`uNM>W7-hRdHVuHd6@tq>p4uvdWy)rF@^V3CVW7m6x>LfNpG
zpa`&W#ewW~6>9azoT(VJAw0B*TMTHBBGgiazz(5}V==k(4OLBhF0+DL4Y^h}=cVP|
zt&WJQ<eD;CE#}w;nUC<B`h&NYE2I&zliplBGv&aYw<Is9m82B;W|3`>^$ovy%ACGj
z0r^p2AHo!mUr#oH^zunUx+V3kEcqS35WhLn=}0ZN-i&KRQt_96i0Gu~A%+*Bv6%k}
z(#sow$)2|E8KGez*EMXJ-kZ6l`_D)K{sK1e$7y3fr^dg^3*O->$&VWGc2qM>wdI#!
zhrK3KlyJj3K7_#>VPiMX`7MdQiby47P<u#pQ!ZfQyXOgBn(1SfC=T0Cq91X%&+&;c
zk$1;{^<Wjt745qN3v2^w-s{+WK%9(v4a%6uX<A>DXsx2LL{{&nYlgIBe|yhJz*N?n
zn7NGFYYXO%9I7`Gf5<kBu`u?ohr`Bm0D1F|V^J0>anO-(nU_A@ns1^EkP+`X;~SHQ
zbXB?+Qkf~&zySayOhGI&1(4fA$`~`UIKbo7?7cKOV$I{Ufq(6~7)bh_qg#gGj5jY{
zu7Hi&@OF$TERmno8xozCT9hN1Mkbjlj`Gl7WWSxVT4X!_uAsA3D0fR>srKV3jH6)C
z@G7hV!~|^!Wah|QZpjlA6=-2NDt0)ua9NKkUjhW7A9ES%!#cKg5zQ0P#F_Sy8Zk@-
zuidjA!(S@0UE97QPqUA1TF<tf=K!n2loAW#Sex0lY#V&06apL=EG@>!)UBNyTXoIF
zasVgMxsl0zaW7Po_(?G?fdhl6Sc8<96+gX%tNZE58YKmYJx=Bg_I%<?ZR?(-5_srZ
z5_Z1=>}UA40b*MziwlrH7iQj+|7(EK&MgybI0c(WOePDt5D9nKRm7xjA(sG#s74}@
zPMV>EfLNyZU@yrFq`(rnGK_c_p^;YXq$+lRl%aw80bqC8as?KmX=~oY5@m8F=v(&u
zpF<u@#Ir=o!8%g@d&3*|BrB#7isB|g9Sjiz1a+LmZlL~TiFyygDFM2Ye$YjB4o1~~
zyf})P605J~a`<?ngW~|nOL2gtc-gRJfV5i0-O|y;GsP0$VRk-0AYi*t5mpe?oB2oc
zvuMU?UuS$_jICtf`=Fp<IH5X15%yFO{P*j|Oh^Fm!^Y_97QZ8L8UlS4;7jT4uce-U
zF$)qXuCqjys0&37iVyBBBPBDZ_^(m|wzPwoDWQ(wdEZq+q=zXF6I-4gH|a<Jty^i4
lT|FUTVUhn2W<9fIH{zv3yR>lly5tWFaCP)N#dIKK{TBtkM$!NP

literal 0
HcmV?d00001

diff --git a/doc/figures/compgraph2.png b/doc/figures/compgraph2.png
new file mode 100644
index 0000000000000000000000000000000000000000..f05f7eeb52189380876350d631919e7ed6038897
GIT binary patch
literal 4557
zcmbtY^;^^59{+5lbhm*tNOuUMLrKTTNeDQ=At4ef3~|JyBm_aEWo&>n62c?}NoB<7
zl8_P*r7z#-x%d77_dd`2oOhpdemJl9`+4567G{R@G(0o_0MHv7=~)2)<uL$&hCpN&
zmRfebl?z5`aT}_Cetv!-TDN@<yI_}mjqCydfR^>21p+ePasdD{(pXO$8cf>EZcUgR
zXYcB1AoBw=Es=Y&L6|R-8}(nKr*&D9a=)vsjj17tgI#U;=$s!``jXfT-tjXgJx>9R
z_O~T6`37AfI2I0rpL6{>FKUtKXO>K^d)lI{bkFba+KJTQr$3I#`3mu!u9dboI9~p5
zY?Lz>aa=tlJegDJ@i3guiTwJIu4nswF^X>6Y6XFbxt~2zOkcEEYqU=x%uw~393_uH
zjZCb@k3Dl1pLZftGPI-S&xzF<j)hZ;`!d{}Q8zP}jW-X>-K^iRuiHW0+pPI4%`|%f
z#R9Vy!yLk@{41~&=qs5%&t1GOe#lz$Mj+mQ980R-P{D5wzTAvZYso%PGTz&odb(;}
z?fq(T&0dmq`cY(BU@dTzTLj+<7)T-6_^kV!eVH+KT80;|MjR%+@|!|@-&lU6t2q%O
zmV1Xmjw<uvVAR&mH(8&c8$DDloNKoJ0oU4>b@3;WGDx=I23EZSF5cwx`1Qyz?d&g5
zMxLtPDQl$FK%`ow$6wp!?W&n8@BN5Gu&0~(-I*Wx=lg|lzP+1SZ1F6$v9B3C^zYwl
z%uFf9wmA3-!@u*xM(kFm<QbpqNO~!22Q(K>C{yGfLEv8sp6Is}vs~WB9~ng-Jfe9d
znY|?kX}UdpYUpTfxh)n;xh`hqhj7qlTZ=143pp!Nhy>EHy`6M1XZOF0rReQ3!!nc?
z={zW*@=5n|QI#9{X#%&l<jv#QaFQc%RZahVo<u#7ceCMk*<1Q5evi^;Ny12df@ehI
zLpEYsJ36$g+8R~Bk*q-Gm_{W&vw4PzqdSo1zTSDIw`aiO_hc69U|8pRIdGe2Clgpr
zFIZI!siOTevGI5?<$!>ciaJbosSIDxtZ@67mk=`<^vk+rNXdD)fFpV|h;-JaOcYq4
zCAQtP8p$8|RqXi&$>vRpjOb&3MZDf97#|bB?m8r#ySZYxbLm~@_mW($obJ?nSXQcW
zkCdm6VfphjX9?_O%901fLrK7cLIkN@7c5?%T78&7K2E~OfNqNqd!SNh3N@kzMZ0$d
z@5}oW1nx(jrr>a^HN3)Wu>k)0agWPh$z7wX%+c^{(HjnHdY_UqH#1!^(EavA&8WI?
zN5?BI`_1z)Dr<tq$X0ZX#pp;bv-^^&9phAig}TV4Zz^iVeRQTz^73=IuA-N;n6Z-&
zGKTkaU8Lj8CTV|OGLaCDDm!R<<LY`k5|v?Y{JM)~;Id49-^Rj=0MV95A#JrcJ-38R
z*>r0E3fT^K%?hWyIAV8g_@S(EY2MQ7W!GVzt?(i8@!d}Qzc}L7?};f#B9!c%Knrzj
zzy&tY>-Bc5wdx#i3fa{?EekSoKJb<&X2Q+xNO^2VHPKiRW%8gV3wnonEwX5(5>4;%
z5Yun@fAq3j)_RT-OT?QlYaA%1&Q?rn{TyF^F-P{TF>JKbdT~U^x~FD7-x}Y=F1~O2
z_4AJ>vB>7@ww@w2UgEr^abgCw5jJ717Y0EI?J6&v*iy8uP8gT$R()4Hx+Dq)F5|1Q
zzw)fe+lXq!2!+A2!Tj!{X6{1z5>+~|*N7XY-7s^+b|*<lA?J!=F!{WZieEj<+;h9L
zKR6yqXF0(|RM;9_7wbRh-}^;58H`5KmDIg}dXM3LX(Ymw3=})Dt}!?`{s&6dptcKp
zJx+Xe1m`D)>A&y1ry4By+E<WN>s4BQY;_0!qsac8Ot2pplAu)TP%(}eTi;2Zw@8Ru
zWjJ)1!~Q63tW|p(9bM-#q1^eSU4Vb|vcu)%07U+3==Apyn&Ti-OQppxG*#S&u+h)%
zWy5$*yfo90^-1*%zZE{zRq(tdr(fa1DnaQ6`417K$tpgMOW3V(5$O$T+py)I#hq5a
zzz#cQdT=VOl7{SgX|r@g{-1jx1PyXkoFcf94MI!mg+RWNNk~7SL@<2r)g}Z{s~8Ye
z!BNhNR)8y1pu+!tKVXOu0xVi1n;UDGnD)G4kjtQ4F@hK3Iq|^^orOygFv9E$0X1Kw
z0eAW5j6kbOWc#8hPHHTF?wNE&<~3(5b17R0dni3f5}f-UQ1F9vX?v{e$|V71F6SuS
zibe?G<fww~Zv+_81Se|~wRg*L3oOa-b7qqqSh!+FNp^n)YYmA&@$yYFTNo0aSk(#4
zHNzo&5>!-f<Ri}coMA?w4p2xXg4Gv=qO^re04r>eje$st1RJ~($do1*4?N=6H%9#+
z|A`ODLfKHqRPQ|skb`t;U==9}Z}02t`_m8r5ZvThkz%R7agjYzVVoA*PzF|VG)=-F
zHxK38HcG*7BiJulSiW2bvG2$ouIv!}jX_+xP%@Ue8w7i{mCxK17D=4V9=n5Uc@!Va
zMYZDU*$6oVljB|yK-|mHL7hQn%1K_uB8-~CSr6psDyPB&vXC7%e+PDS3?_9x&DU2J
zn>(5^Q(N24j#o<1jH#a#>f?|(7^Jgkt~)Q8(3<&FSH%`z5I{D?;uZDrc=QWs>^>zT
z#wJRM>FAhzN4P;P%m;N;ufxlA^yF862$-P`_1z}w3=Up>x5j)MnB_#_1I&iWWmL2X
zo7e75aiDlNHvlL4x`I^>nJnS_05Y~ou`t@;bCOJs2<mJi&1<{vcWaqejPheEoaw+e
zc(u#c(`?7eX>uhL9W)dGJq1W4t**6+iC!;%SqH(fH$~X|oav~?lH$xKqdOwc>|TL3
zfA9ZpJr6yGRb?7v{YpH)+*gEMh$ha?NQQAkEn~;e&e}h;weGv?Q#S9O?fX1CcQ%0y
zwuEn<3>rzvr&>;b;*o6SZLorJj_%;cv$gi!izsW@#MBtCJlwu2ErfS_s~GpNya-G8
zeb?aLpN3Q$fic>Zm&YxoBPcTzYWQ@q?0oeWY~gdTJk!IaSD%%-XJkHY=f!y~NVVC8
z&gx3T_V@B#PJ&moqN<4SlP;F<KSkIZPZmQ)<N4B*;pn{j^{*H@ELFtEz1H+@30=f%
zp}SvoXTOZpS*rR&18gP@e&<U|$mJT9wbi=07sG;Hu@MF1dtVz+N}3d89=PjM?cc%2
zy15s@TKLk^4TP4kZ3^N||77H$TN`M@xNgA&=36T}$7scH4T1BfrB2b-(9k(J`qdxO
zb`ORQ;bairLLXKki0_f!n&n#XHphq`EfRR`1`Dwrr*2#WpVV7LE=~(<2>wWaF5nQN
z6<!n9B%zAiVVn)VSIt4(9H-|j=d&(`xe+QGrWkC3{QE2ZPIs|<SlZQPoGsJv7EQm}
zm0i85Tpis~`UN!*>J|Ncza5Sy%GsNr2KuKvb6cM<A@x$HYM=(}lZ+DT<NNM`eH26T
z_ENIf$mIS@b1Q&~mO7z2oaOnQ^MaiXfKe1>gBV;0I<9y9Z*BTtg_9AGR{F#D$P@H1
z-QNbXuuLbxr^(XD`-DB<kxXs+(5*)sLX=^W{$nt@jAbf&(_xwd7B1@*m;&VXrdZf-
zg##AmKMy-mo>UMCs<PLhgQBxC1S*4M7GtR`J*<4{AnQy5X{JBSt<M6xdQE=h%34VH
zc~-_Hvk4&FbJHT#TUKW=lrbepXKC)mgqIhdom+r{s|tFq!-7$IaXQpCah^?Q1tyrr
z7?vd79;_U-#800!%`Vn4RLfpAGNZ=(TluX|cJ>jk@W59RFUIuSZrBG{4Ozz44MxI*
z4P>l$cK`UDKC;RrsIow_3~^KDd86o)*8ycx_BydeAM!fZr!bFOu0JW}T`NkTI5*x3
zJ)2HVZhCd<b9`zyybs3Z=D)~ry;K?Hr6+}LtW)!1NTn%ot$RgNAUkSHi>*|v&kxkT
zI@N7UCzZ=}qu8ZbOCq_;7Jg$cUG4?0QB)U;ds$wFuqRAHw>7$0JVBE<t!)zK{VwXK
zes7FiP@Mm|`1w=liJX{I<{x)`ho6)vuf;BLa$X%#<;OmOQ(Og}SKdRYJ#)hQv5xBw
z7oZn>*4~BawlaCsTV=#;iFUp~_cv<0qSH^=2^JKhfiJETF78V{vvsD-80zYrX0qbt
z4AOms)K3|aX=Frd5d6HUk&aK>AB@2i&_hrCd1)WqWBuN-*^Pn4QvRc_^o!L00@RNx
zu1b&;R&^Wgzev|tMj$oTyD>`i3n=+Ex_7!MBQxz6dZB&z<zBecKqB{&q2w*fQ`^hx
zgPhqdN?$*YyMhE?t#)GsSv}!>iNLO|v*FehrjY-~PrAA>!!m9`%eqhAcuEwdeY^}#
z)qs89ML$I%XP(81WzC9a#rToHrt9c{FNGBgJkmO><xX9%jYhJyUrwA`T23U+?y!5*
z>rL9S$>nQFhF-TY;*6hD^6hUu>nT%w!-(P-mA?orPH}S&ab8BNP+x{UUg~!tf_4S{
z3mK9`hSghp*B?;HzIOC-FMN=egG(zjAeE;vN2aNQC4wG5kj>nb(UAb5j^Gu%#3D3`
z*_OVtOaoA$u%$Px$n4JoiI=jW)W8g)s4;34DuM(#pBONG4^+MJtIw-~NfBaMbfV9I
zFo|})_PeRi<ebyilz_|FVsuyz5pqD!RDO+Z-nLIpqeOX>b*1GH9-3G#f92sUeScP-
zCPmgrondFETd~gPOKl0i<{Wk}^FJ;7h>1v^-`IhTV*>7($SocBz%217Fi4pxE%xFu
zGbvS<K`WbJ*So4rL?%!zV$9?(KZOh!-<fYp0D*a!h_zr}Gf>BthV5K%trzQ1A<#$z
z;j~I6N0<?M-_&0UMx(&_nLcKewpmoN+zDU>G1OQw86DG5=T@BMvf{unN|z<~7HDJ&
z25R>}VcaYL&2WQ^Y-^N!#dS~ulLcb}MX7W;ShSA*l0a*v7xZ^VgroZ462u*E2f$n^
zysaIO#Ft1_@vG(%R=^&`2gmvqo7E+v9v%MDqGY-7(7H&)x0dHc?v+h$?(^cr(M-wo
z-ORkYIkYv-0x)1B2uY>$#RH<g%ZjQzL!Ec*L2tjY1i$O;@K`6|K)D3@bYX*KSE%+?
zm_i7G-4&{-=k5Z9JN$}u(m<;v4W@swq+m*muna^J=SVMNr@wVFFwIe*!%B^K=H~g@
zdVU`%egLfcFTQJGbt_=cOLWwbzCy5-*tywv5rbQ%ddK-8C+oHHVD4a$Phg+F>#V(=
z<{<P|zP;9;qe`_OBae}i3%Z=OzOKJ227{3L-}!h9w)Q#HLRBK7E-(Sda|c@(X0QeX
zr!0JGs>|<bbGq5Vm=|9JdFiBHulD%*eno_i(bBHeD4?@CwLTu}%Rp88)SHo4@I~C(
z6h?_TYR4f^%kS{toA>TTQ_SS>y&(}qd-Eps2Q>ihFbticaGD;y^0{u^;1Bg#GikJz
z-Bf8y88o<CFaBazxk*ccB+5z%Q+i?1Xv3cxuC+FZd4J*~w5AL1Ofy{DbLliZE==58
zQP%9qNn5`tPhwm3QB0VCR4eL3!{a6Mra+|r+jZTLKYgR=*L(J@5(HoR3K@P&WJe0$
zmJTLr@?_oYDLS~Fx^X*b^Um;^u8UVr)#el?qkvz-Pqab4{H!ByDRNHToX<p)x@B)C
zzq~AF5y7{qi5p0WoHity-V27pn`fRu;TLU1_3vA>kVXb1nu=AB@!t3|8SOFHNYB~S
kRrRj_XMn+hcR#(lY|%)FBSs5eOk)6JeKWms9p~r&0yU?F?*IS*

literal 0
HcmV?d00001

diff --git a/doc/figures/compgraph3.png b/doc/figures/compgraph3.png
new file mode 100644
index 0000000000000000000000000000000000000000..92309630483a9e1da3dc03441fa0b24f14dec0f1
GIT binary patch
literal 2645
zcmV-b3aa&qP)<h;3K|Lk000e1NJLTq00K(@00QL!0{{R3?qx9F00006P)t-s|Ns90
z0033O(|!N|010qNS#tmY3ljhU3ljkVnw%H_015R;L_t(|+U=daZsSN2$D3}0eE~rZ
zun7Y~Jm6k{mnjg2^C&vp3ka920%Jr&z<Ps$C%s^Rdyxxh7?{mw(+v)Hob>~&2Ae1l
z#4|rcHPxid?y6aHz4ZsiBTMGb-OZ}%4|bOsBauiX5{X12kw_#Gi9{liNF)-8^v9Kz
zhy)2G5|Lt|9{+KSa`EabM!CGIJQ%1;{QJnkhHykdtLPnHBBDkrj)+P@-q*UD>5Fdk
z@k*EnVl;E;i0H|0T^$m(9HXn=;5m4P7XCodsju5YviKpM`8*N+#4|HAl!u3kH_wRj
zdr@0m)Uxk`)lL^R@B1L%p;>6YN3&2&RE}4{9L)(=cqlJV@oK1tIv00Wdj5Q_kN5PP
z&^=e;r=2j)TPlBd`>nLHD_LJ16?J^uzFDbqLpY%bMW{X$&lFTW%F;%;k%AJ2D92Y&
zp&=>=6cjf^`FN<QODID9QBb~vh61_sW$~@wpwuA42S-t-_%?1r-xLt~f)bVW2jnn&
z0(EsUWfzMX+r~+O+L?C~)~9gKCfi^5hYFP^#wdRC{$I@dvcB=Ew#qlgD8X;M$1m$o
zqDJijsE>l#!ullMG4{F`oSLJo8pSbCJpQgmMR3$d3AF{GnhF-E6xvqT2h`r&M8)-Y
zk0^fqBd?5MV~*nM8}|`)cO%?41u6kJ&JW@P^Q;Gc>lKbOgACR7V#~txt+n&uqdpsN
z0NH{16Z6<VCu}33{>)|<^JTHTrib!&@y)-#z5XDf;>~{Y<-hhqyHwM^{jj^ccjQui
zz6pO7PIMOKb+py;?&13G;Zj0b58_sccdyQ(!c;|V?jF|3Ln5L0y|{Jxt$sh4<pnB}
zQS1B>^*+%>Sy{^0$!GZviZydxev|5=cv_&corLmufm(00QAvq<Euq{Jm27oUBFX-;
zPJf<As0mMQJ^uYt7v)`U{(hbQ>`JKFH+Q#!e?QelxtH7f>+D`eO}}|?#oEzEO`O$z
z@@4%_LM{KAyZoaY>7v42>}7GeJaj;<HoG|473<P8cP12}eh;Vx0EM_(8@7Ka1E89<
zd39+%)mo50d*K*Rj8KFk)U%?TQKIZoqR6fYMW|;+&GAstOGlAi5sFYIs00mVA)->e
z3Z`RhtFLGb>i#?9?Th0?#p-f`=~mBiqC&M{R@;|^^3;YIyc^m6YAB(i!l9@WXBBG0
z)Q|grQ`9M0MVW3*=W4@v>9eCO6=iy4(o!47^dRRbQ6kSzts$&eb9L1$10Qc`@s>fk
z!bcAj?J%=?yrrQ*m?@)iEZ=;-Yu<la(h85*D85rY2Oj7<5~|+q_*(k(ZS~()7qwG7
z<={uLX6!p`O_H3aK8maDJdG`0HE&J&UGuCa%8TQfk)<ALsTA+%MiV{MMj4<UPaNqT
z<&#b-9iU#<hdQm!*i@~SmP<?L3xX3QtDis(RiIL(>!&&>>4$|{t3U^}P^K&LRKwOm
zWy<JXK2bkt^zL`ecIJvP4LToJqgsdh7pM1h*ulV{!=9&VQ)j~imaUewb(U;yWT6Lq
z%a)?yC`aW+J!Uts_2t^mdp{p+?1y`h_4w6OisbmtB}1((%F#vDX8|fBOXc9>4o6iD
zQ=bJWOHvo7?j3DiRK0mvMsY(_eHNf}lQv4|28Px~>lR95VxOugrj2@qhw|}I0UpZ1
zLoq@bp|sBe^rN)T0-T8qzUt1}gS84(p9PSEj4%u`bwF-~zDyU@>Z#;_+|^I&pqPRh
z)=l_I=f%~|YZxlP>rx%1%XHPDtVT!gsNY0}!w*}HhJMdb^~o6k!cdKM3?L)blr&~0
zmO2)-b;qJzzcy2XfmSN3OeunVxz;Rhp%g)`l&gb6oa0Q?m&=aX@DsW`X^oj$l#Pzl
zNR7Nu8fdUB*zM_i^XfPU9=slAx0{2jivwCW@Yq7qM1G@uHcXy+-AuRDPuCJQ3SB!}
z%2rtxIxHTAy6bbXdfo1H?PzFYm1VHx)%!R(uH5O!+8==jd)gj>dNte>&$wD|6<cG4
z!9yXQZ$;j&jF)GjQFe?l&-d+K%2}yv!RL$9u=Xe2aG?80NDjR|Gn_K3F~nhp{d=vX
z4;3+hY-ZTMKZTk@P@EYK6`mM8z;Y9|1`n_dQLWAem~1NTZ8&XZc&K@=WoTc&0HUI9
zUvXflDG=2YS9};Mz(YA$F{V9g0itZI7}FkCXD}4UiZQK#WtKDVvMaEd#TdJ*C}v4|
zMOOoHXk%w)lce(j$A+R7X7jD{0mp%&e6y+Bd6W-DIe;NiFDinfZ19k1p`uK~)l@Zn
z({S}lMWxWTYC?;Jh)STSrd|j+#WS2Ck!%At9JRnhMc^32PmL9TN={;nd5mc+GJuDx
z<fLK|yxCnml#MeaDtpqZE8QTVc(s<YOA%0cvs?j=G6<;JkVK7w3?GBTB1nbW!VRk4
z*bGYRdMTrz$!0H#FEI*ARBctEVA*ubEoEMff+aC7_X&s^1)EEh8~J#Rf?0c<H4qD!
z0e7`&T*d;X_^nW@vYNoe!<Dm?TPis)vvOriRpW1g;iX!!Em1JDsxvc}i`zT{qRQJ=
zQh+JRMYWN(d?*3@OEQn8`d=$61EA?o5=uZ&{N!TDEGNc8aXi$?Yzu_)2}P)ri?7g7
zhwcUe#oyzh>cEG9y5MLi8xO@sio!Edfrqm3P!1l-$3sP<M9uL~=^MOwm7)b1Je0`s
ze31G<LX8_0;GwQYS*oB&e~X5i<DoJf6#tL7+u@+xtKc&dDws`|NNwc@izyOn8Z2f=
zs3=$jc%=$nbg2+gGk@vN(M)vu4;)mu4;EKADDjh*EvfS&6rl)pP>8i!i;BXi8KDS8
zC_)j6P=q2Bp$J7NLJ^8kFCXOr_fn1?C4f6TM~~tdyR65Ig6@X>Iw*#TVq-+D@KA3@
ziF%EPn!Rw;lYfG3u!_~+9SW)3c8?Qg%Jf+RV=0>L(R`=gjxG>tSZ7AG5NddZ=R8f`
zT5u$7_Q-Q^V|C=-G1W1T68N<2vI7cib@(Z4@G*1Nbv^W8Dq^Al#7qc?sU8q>SuA=g
zYoM&>78po20FGs$;3P0fA6MVldut0acA&Z2t@u8NqYh^QKzWKP13#Pvh!9ZqSpWpo
z;Vi%u9vJ8>fCh@r0t`pdS%AUrd^ig*MF=uA3L(fG&H}u^#Bl--h1jKXypA3*2VHwm
z1VbSX)u7qldIs3Qv8dlO@CJ_CfwMRl%LQ?hp`~3fp@s!KX^xtet0M(2d~sj_MRoiw
zA>%NUVYT{kp#i+ia7@jMu;?VJ8uAQ|s?+?&@TRJBSy(8mndM(Ci!%%qTjdOxx8)cT
z@*M3!4&oyr5{X12kw_#Gi9{liNF)-8L?V$$zaRY{<C}eFy7v5u00000NkvXXu0mjf
D!N~1T

literal 0
HcmV?d00001

diff --git a/doc/introduction.md b/doc/introduction.md
new file mode 100644
index 000000000..d1900b9b4
--- /dev/null
+++ b/doc/introduction.md
@@ -0,0 +1,49 @@
+## Introduction
+
+The goal of Ray is to make it easy to write machine learning applications that
+run on a cluster while providing the development and debugging experience of
+working on a single machine.
+
+Before jumping into the details, here's a simple Python example for doing a
+Monte Carlo estimation of pi (using multiple cores or potentially multiple
+machines).
+```python
+import ray
+import functions # See definition below
+
+results = []
+for _ in range(10):
+  results.append(functions.estimate_pi(100))
+estimate = np.mean([ray.get(ref) for ref in results])
+print "Pi is approximately {}.".format(estimate)
+```
+
+This assumes that we've defined the file `functions.py` as below.
+```python
+import ray
+import numpy as np
+
+@ray.remote([int], [float])
+def estimate_pi(n):
+  x = np.random.uniform(size=n)
+  y = np.random.uniform(size=n)
+  return 4 * np.mean(x ** 2 + y ** 2 < 1)
+```
+
+Within the for loop, each call to `functions.estimate_pi(100)` sends a message
+to the scheduler asking it to schedule the task of running
+`functions.estimate_pi` with the argument `100`. This call returns right away
+without waiting for the actual estimation of pi to take place. Instead of
+returning a float, it returns an **object reference**, which represents the
+eventual output of the computation.
+
+The call to `ray.get(ref)` takes an object reference and returns the actual
+estimate of pi (waiting until the computation has finished if necessary).
+
+## Next Steps
+
+- [Download and Setup](download-and-setup.md)
+- [Basic Usage](basic-usage.md)
+- [Tutorial](tutorial.md)
+- [About the System](about-the-system.md)
+- [Using Ray on a Cluster](using-ray-on-a-cluster.md)
diff --git a/doc/tutorial.md b/doc/tutorial.md
new file mode 100644
index 000000000..4feb74f16
--- /dev/null
+++ b/doc/tutorial.md
@@ -0,0 +1,118 @@
+## Tutorial
+
+This section assumes that Ray has been built. See the [instructions for
+installing Ray](download-and-setup.md)
+
+### Trying it out
+
+Start a shell by running this command.
+```
+python scripts/shell.py
+```
+By default, this will start up several things
+
+- 1 scheduler (for assigning tasks to workers)
+- 1 object store (for sharing objects between worker processes)
+- 10 workers (for executing tasks)
+- 1 driver (for submitting tasks to the scheduler)
+
+Each of the above items, and each worker, is its own process.
+
+The shell that you just started is the driver process.
+
+You can take a Python object and store it in the object store using `ray.put`.
+This turns it into a **remote object** (we are currently on a single machine,
+but the terminology makes sense if we are on a cluster), and allows it to be
+shared among the worker processes. The function `ray.put` returns an object
+reference that is used to identify this remote object.
+
+```python
+>>> xref = ray.put([1, 2, 3])
+```
+
+We can use `ray.get` to retrieve the object corresponding to an object
+reference.
+
+```python
+>>> ray.get(xref)
+[1, 2, 3]
+```
+We can call a remote function.
+```python
+>>> ref = example_functions.increment(1)
+>>>ray.get(ref)
+2
+```
+
+Note that `example_functions.increment` is defined in
+[`scripts/example_functions.py`](../scripts/example_functions.py) as
+
+```python
+@ray.remote([int], [int])
+def increment(x):
+  return x + 1
+```
+
+Note that, we can pass arguments into remote functions either by value or by
+object reference. That is, these two lines have the same behavior.
+
+```python
+>>> ray.get(example_functions.increment(1))
+2
+>>> ray.get(example_functions.increment(ray.put(1)))
+2
+```
+This is convenient for chaining remote functions together, for example,
+```python
+>>> ref = example_functions.increment(1)
+>>> ref = example_functions.increment(ref)
+>>> ref = example_functions.increment(ref)
+>>> ray.get(ref)
+4
+```
+
+### Visualize the computation graph
+
+At any point, we can visualize the computation graph by running
+```python
+>>> ray.visualize_computation_graph(view=True)
+```
+This will display an image like the following one.
+
+<p align="center">
+  <img src="figures/compgraph1.png" width="300">
+</p>
+
+### Restart workers
+
+During development, suppose that you want to change the implementation of
+`example_functions.increment`, but you've already done a bunch of work in the
+shell loading and preprocessing data, and you don't want to have to recompute
+all of that work.
+
+We can simply restart the workers.
+
+First, change the code, for example, modify the function
+`example_functions.increment` in
+[`scripts/example_functions.py`](../scripts/example_functions.py) to add 10
+instead of 1.
+
+```python
+@ray.remote([int], [int])
+def increment(x):
+  return x + 10
+```
+Then from the shell, restart the workers like this.
+```python
+>>> ray.restart_workers("scripts/example_worker.py") # This should be the correct relative path to the example_worker.py code
+```
+We can check that the code has been updated by running.
+```python
+>>> ray.get(example_functions.increment(1))
+11
+```
+
+Note that it is not as simple as running `reload(example_functions)` because we
+need to reload the Python module on all of the workers as well, and the workers
+are separate Python processes. Calling `reload(example_functions)` would only
+reload the module on the driver.
diff --git a/doc/using-ray-on-a-cluster.md b/doc/using-ray-on-a-cluster.md
new file mode 100644
index 000000000..e20dabcfa
--- /dev/null
+++ b/doc/using-ray-on-a-cluster.md
@@ -0,0 +1,79 @@
+## Using Ray on a cluster
+
+Running Ray on a cluster is still experimental.
+
+Ray can be used in several ways. In addition to running on a single machine, Ray
+is designed to run on a cluster of machines. This document is about how to use
+Ray on a cluster.
+
+### Getting started with Ray on a cluster
+
+These instructions work on EC2, but they may require some modifications to run
+on your own cluster. In particular, on EC2, running `sudo` does not require a
+password, and we currently don't handle the case where a password is needed.
+
+1. Create a file `nodes.txt` of the IP addresses of the nodes in the cluster.
+For example
+
+        12.34.56.789
+        12.34.567.89
+The first node in the file is the "head" node. The scheduler will be started on
+the head node, and the driver should run on the head node as well.
+
+2. Make sure that the nodes can all communicate with one another. On EC2, this
+can be done by creating a new security group and adding the inbound rule "all
+traffic" and adding the outbound rule "all traffic". Then add all of the nodes
+in your cluster to that security group.
+
+3. Run something like
+
+    ```
+    python scripts/cluster.py --nodes nodes.txt \
+                              --key-file key.pem \
+                              --username ubuntu \
+                              --installation-directory /home/ubuntu/
+    ```
+where you replace `nodes.txt`, `key.pem`, `ubuntu`, and `/home/ubuntu/` by the
+appropriate values. This assumes that you can connect to each IP address
+`<ip-address>` in `nodes.txt` with the command
+    ```
+    ssh -i <key-file> <username>@<ip-address>
+    ```
+4. The previous command should open a Python interpreter. To install Ray on the
+cluster, run `install_ray()` in the interpreter. The interpreter should block
+until the installation has completed.
+5. To check that the installation succeeded, you can ssh to each node, cd into
+the directory `ray/test/`, and run the tests (e.g., `python runtest.py`).
+6. Now that Ray has been installed, you can start the cluster (the scheduler,
+object stores, and workers) with the command
+`start_ray("/home/ubuntu/ray/scripts/default_worker.py")`, where the second
+argument is the path on each node in the cluster to the worker code that you
+would like to use. After completing successfully, this command will print out a
+command that can be run on the head node to attach a shell (the driver) to the
+cluster. For example,
+
+    ```
+    source "$RAY_HOME/setup-env.sh";
+    python "$RAY_HOME/scripts/shell.py" --scheduler-address=52.50.28.103:10001 --objstore-address=52.50.28.103:20001 --worker-address=52.50.28.103:30001 --attach
+    ```
+
+7. Note that there are several more commands that can be run from within
+`cluster.py`.
+
+    - `install_ray()` - This pulls the Ray source code on each node, builds all
+      of the third party libraries, and builds the project itself.
+    - `start_ray(worker_path, num_workers_per_node=10)` - This starts a
+      scheduler process on the head node, and it starts an object store and some
+      workers on each node.
+    - `stop_ray()` - This shuts down the cluster (killing all of the processes).
+    - `restart_workers(worker_path, num_workers_per_node=10)` - This kills the
+      current workers and starts new workers using the worker code from the
+      given file. Currently, this can only run when there are no tasks currently
+      executing on any of the workers.
+    - `update_ray()` - This pulls the latest Ray source code and builds it.
+
+### Running Ray on a cluster
+
+Once you've started a Ray cluster using the above instructions, to actually use
+Ray, ssh to the head node (the first node listed in the `nodes.txt` file) and
+attach a shell to the already running cluster.
diff --git a/scripts/example_functions.py b/scripts/example_functions.py
index e3e43f24f..f04e55a60 100644
--- a/scripts/example_functions.py
+++ b/scripts/example_functions.py
@@ -12,6 +12,10 @@ def estimate_pi(n):
 def increment(x):
   return x + 1
 
+@ray.remote([int, int], [int])
+def add(a, b):
+  return a + b
+
 @ray.remote([List[int]], [np.ndarray])
 def zeros(shape):
   return np.zeros(shape)