From 8d5e61d3c0969ddf365f992c5befef3c7b76a8d1 Mon Sep 17 00:00:00 2001
From: Robert Nishihara <robertnishihara@gmail.com>
Date: Thu, 28 Jul 2016 20:47:37 -0700
Subject: [PATCH] update tutorial (#318)

---
 README.md                     |  41 ++---
 doc/about-the-system.md       |  12 +-
 doc/basic-usage.md            | 207 -----------------------
 doc/figures/compgraph1.png    | Bin 2819 -> 4914 bytes
 doc/figures/compgraph2.png    | Bin 4557 -> 2645 bytes
 doc/figures/compgraph3.png    | Bin 2645 -> 0 bytes
 doc/install-on-macosx.md      |  10 +-
 doc/install-on-ubuntu.md      |  10 +-
 doc/install-on-windows.md     |   4 +-
 doc/remote-functions.md       |  79 +++++++++
 doc/tutorial.md               | 307 +++++++++++++++++++++++++---------
 doc/using-ray-on-a-cluster.md |  12 +-
 examples/hyperopt/README.md   |   8 +-
 examples/lbfgs/README.md      |   6 +-
 14 files changed, 357 insertions(+), 339 deletions(-)
 delete mode 100644 doc/basic-usage.md
 delete mode 100644 doc/figures/compgraph3.png
 create mode 100644 doc/remote-functions.md

diff --git a/README.md b/README.md
index 1bfbe9250..4cc3d939d 100644
--- a/README.md
+++ b/README.md
@@ -2,8 +2,8 @@
 
 [![Build Status](https://travis-ci.org/amplab/ray.svg?branch=master)](https://travis-ci.org/amplab/ray)
 
-Ray is an experimental distributed execution framework with a Python-like
-programming model. It is under development and not ready for general use.
+Ray is an experimental distributed extension of Python. It is under development
+and not ready for general use.
 
 The goal of Ray is to make it easy to write machine learning applications that
 run on a cluster while providing the development and debugging experience of
@@ -12,35 +12,37 @@ working on a single machine.
 Before jumping into the details, here's a simple Python example for doing a
 Monte Carlo estimation of pi (using multiple cores or potentially multiple
 machines).
-```python
-import ray
-import functions # See definition below
 
-results = []
-for _ in range(10):
-  results.append(functions.estimate_pi(100))
-estimate = np.mean([ray.get(ref) for ref in results])
-print "Pi is approximately {}.".format(estimate)
-```
-
-This assumes that we've defined the file `functions.py` as below.
 ```python
 import ray
 import numpy as np
 
+# Start a scheduler, an object store, and some workers.
+ray.services.start_ray_local(num_workers=10)
+
+# Define a remote function for estimating pi.
 @ray.remote([int], [float])
 def estimate_pi(n):
   x = np.random.uniform(size=n)
   y = np.random.uniform(size=n)
   return 4 * np.mean(x ** 2 + y ** 2 < 1)
+
+# Launch 10 tasks, each of which estimates pi.
+results = []
+for _ in range(10):
+  results.append(estimate_pi(100))
+
+# Fetch the results of the tasks and print their average.
+estimate = np.mean([ray.get(ref) for ref in results])
+print "Pi is approximately {}.".format(estimate)
 ```
 
-Within the for loop, each call to `functions.estimate_pi(100)` sends a message
-to the scheduler asking it to schedule the task of running
-`functions.estimate_pi` with the argument `100`. This call returns right away
-without waiting for the actual estimation of pi to take place. Instead of
-returning a float, it returns an **object reference**, which represents the
-eventual output of the computation.
+Within the for loop, each call to `estimate_pi(100)` sends a message to the
+scheduler asking it to schedule the task of running `estimate_pi` with the
+argument `100`. This call returns right away without waiting for the actual
+estimation of pi to take place. Instead of returning a float, it returns an
+**object reference**, which represents the eventual output of the computation
+(this is a similar to a Future).
 
 The call to `ray.get(ref)` takes an object reference and returns the actual
 estimate of pi (waiting until the computation has finished if necessary).
@@ -48,7 +50,6 @@ estimate of pi (waiting until the computation has finished if necessary).
 ## Next Steps
 
 - Installation on [Ubuntu](doc/install-on-ubuntu.md), [Mac OS X](doc/install-on-macosx.md), [Windows](doc/install-on-windows.md)
-- [Basic Usage](doc/basic-usage.md)
 - [Tutorial](doc/tutorial.md)
 - [About the System](doc/about-the-system.md)
 - [Using Ray on a Cluster](doc/using-ray-on-a-cluster.md)
diff --git a/doc/about-the-system.md b/doc/about-the-system.md
index 52b7eb164..9bc2611a5 100644
--- a/doc/about-the-system.md
+++ b/doc/about-the-system.md
@@ -1,9 +1,9 @@
-## About the System
+# About the System
 
 This document describes the current architecture of Ray. However, some of these
 decisions are likely to change.
 
-### Components
+## Components
 
 A Ray cluster consists of several components.
 
@@ -12,20 +12,20 @@ A Ray cluster consists of several components.
 - One object store per node
 - One (or more) drivers
 
-#### The scheduler
+### The scheduler
 
 The scheduler assigns tasks to the workers.
 
-#### The workers
+### The workers
 
 The workers execute tasks and submit tasks to the scheduler.
 
-#### The object store
+### The object store
 
 The object store shares objects between the worker processes on the same node so
 that the workers don't need to each have their own copies of the objects.
 
-#### The driver
+### The driver
 
 The driver submits tasks to the scheduler. If you use Ray in a script, the
 Python process running the script is the driver. If you use Ray interactively
diff --git a/doc/basic-usage.md b/doc/basic-usage.md
deleted file mode 100644
index feeeb09e2..000000000
--- a/doc/basic-usage.md
+++ /dev/null
@@ -1,207 +0,0 @@
-## Basic Usage
-
-To use Ray, you need to understand the following:
-
-- How Ray uses object references to represent immutable remote objects.
-- How Ray constructs computation graphs using remote functions.
-
-### Immutable remote objects
-
-In Ray, we can create and manipulate objects. We refer to these objects as
-**remote objects**, and we use **object references** to refer to them. Remote
-objects are stored in **object stores**, and there is one object store per node
-in the cluster. In the cluster setting, we may not actually know which machine
-each object lives on.
-
-An **object reference** is essentially a unique ID that can be used to refer to
-a remote object. If you're familiar with Futures, our object references are
-conceptually similar.
-
-We assume that remote objects are immutable. That is, their values cannot be
-changed after creation. This allows remote objects to be replicated in multiple
-object stores without needing to synchronize the copies.
-
-#### Put and Get
-
-The commands `ray.get` and `ray.put` can be used to convert between Python
-objects and object references, as shown in the example below.
-```python
->>> x = [1, 2, 3]
->>> ray.put(x)
-<ray.ObjRef at 0x1031baef0>
-```
-
-The command `ray.put(x)` would be run by a worker process or by the driver
-process (the driver process is the one running your script). It takes a Python
-object and copies it to the local object store (here *local* means *on the same
-node*). Once the object has been stored in the object store, its value cannot be
-changed.
-
-In addition, `ray.put(x)` returns an object reference, which is essentially an
-ID that can be used to refer to the newly created remote object. If we save the
-object reference in a variable with `ref = ray.put(x)`, then we can pass `ref`
-into remote functions, and those remote functions will operate on the
-corresponding remote object.
-
-The command `ray.get(ref)` takes an object reference and creates a Python object
-from the corresponding remote object. For some objects like arrays, we can use
-shared memory and avoid copying the object. For other objects, this currently
-copies the object from the object store into the memory of the worker process.
-If the remote object corresponding to the object reference `ref` does not live
-on the same node as the worker that calls `ray.get(ref)`, then the remote object
-will first be copied from an object store that has it to the object store that
-needs it.
-```python
->>> ref = ray.put([1, 2, 3])
->>> ray.get(ref)
-[1, 2, 3]
-```
-
-If the remote object corresponding to the object reference `ref` has not been
-created yet, the command `ray.get(ref)` will wait until the remote object has
-been created.
-
-### Computation graphs in Ray
-
-Ray represents computation with a directed acyclic graph of tasks. Tasks are
-added to this graph by calling **remote functions**.
-
-For example, a normal Python function looks like this.
-```python
-def add(a, b):
-  return a + b
-```
-A remote function in Ray looks like this.
-```python
-@ray.remote([int, int], [int])
-def add(a, b):
-  return a + b
-```
-
-The information passed to the `@ray.remote` decorator includes type information
-for the arguments and for the return values of the function. Because of the
-distinction that we make between *submitting a task* and *executing the task*,
-we require type information so that we can catch type errors when the remote
-function is called instead of catching them when the task is actually executed.
-
-However, the only piece of information that is fundamentally required by the
-system is the number of return values (because the system must assign the
-correct number of object references to the outputs before the function has
-actually executed and produced any outputs).
-
-#### Remote functions
-
-Whereas in regular Python, calling `add(1, 2)` would return `3`, in Ray, calling
-`add(1, 2)` does not actually execute the task. Instead, it adds a task to the
-computation graph and immediately returns an object reference to the output of
-the computation.
-
-```python
->>> ref = add(1, 2)
->>> ray.get(ref)
-3
-```
-
-There is a sharp distinction between *submitting a task* and *executing the
-task*. When a remote function is called, the task of executing that function is
-submitted to the scheduler, and the scheduler immediately returns object
-references for the outputs of the task. However, the task will not be executed
-until the scheduler actually schedules the task on a worker.
-
-When a task is submitted, each argument may be passed in by value or by object
-reference. For example, these lines have the same behavior.
-
-```python
->>> add(1, 2)
->>> add(1, ray.put(2))
->>> add(ray.put(1), ray.put(2))
-```
-
-Remote functions never return actual values, they always return object
-references.
-
-When the remote function is actually executed, it operates on Python objects.
-That is, if the remote function was called with any object references, the
-Python objects corresponding to those object references will be retrieved and
-passed into the actual execution of the remote function.
-
-#### Blocking computation
-
-In a regular Python script, the specification of a computation is intimately
-linked to the actual execution of the code. For example, consider the following
-code.
-```python
-result = []
-for i in range(10):
-  result.append(np.zeros(size=[100, 100]))
-```
-
-At the core of the above script, there are 10 separate tasks, each of which
-generates a 100x100 matrix of zeros. These tasks do not depend on each other, so
-in principle, they could be executed in parallel. However, in the above
-implementation, they will be executed serially.
-
-Ray gets around this by representing computation as a graph of tasks, where some
-tasks depend on the outputs of other tasks and where tasks can be executed once
-their dependencies have been executed.
-
-For example, suppose we define the remote function `zeros` to be a wrapper
-around `np.zeros`.
-```python
-from typing import List
-import numpy as np
-
-@ray.remote([List[int]], [np.ndarray])
-def zeros(shape):
-  return np.zeros(shape)
-```
-Then we can write
-```python
-result_refs = []
-for i in range(10):
-  result.append(zeros([100, 100]))
-```
-This adds 10 tasks to the computation graph, with no dependencies between the
-tasks.
-
-The computation graph looks like this.
-
-<p align="center">
-  <img src="figures/compgraph2.png">
-</p>
-
-In this figure, boxes are tasks and ovals are objects.
-
-The box that says `op-root` in it just refers to the overall script itself. The
-dotted lines indicate that the script launched 10 tasks (tasks are denoted by
-rectangular boxes). The solid lines indicate that each task produces one output
-(represented by an oval).
-
-It is clear from the computation graph that these ten tasks can be executed in
-parallel.
-
-Computation graphs encode dependencies. For example, suppose we define
-```python
-ray.remote([np.ndarray, np.ndarray], [np.ndarray])
-def dot(a, b):
-  return np.dot(a, b)
-```
-Then we run
-```python
-aref = zeros([10, 10])
-bref = zeros([10, 10])
-cref = dot(aref, bref)
-```
-The corresponding computation graph looks like this.
-
-<p align="center">
-  <img src="figures/compgraph3.png" width="300">
-</p>
-
-
-The three dashed lines indicate that the script launched three tasks (the two
-`zeros` tasks and the one `dot` task). Each task produces a single output, and
-the `dot` task depends on the outputs of the two `zeros` tasks.
-
-This makes it clear that the two `zeros` tasks can execute in parallel but that
-the `dot` task must wait until the two `zeros` tasks have finished.
diff --git a/doc/figures/compgraph1.png b/doc/figures/compgraph1.png
index 9abd43a362b2a9716c915d1e42ddde6daaf2f2ee..bc34726b07f488ab7518f30e83e9ab5e7003ff41 100644
GIT binary patch
literal 4914
zcmbtYhd12a)8DTZEvtoPwXj-NFChp*u<AoBYqekzHEN=SD2s?*V<RD=vsP!d=rxHN
zVbu^RL`{$&BzX7tyyyJ~e(yPR&)oT(J9F-tJ9B5wB$%4$F*0y7fIuKd1AQGc5Qydv
z2n4(Xz!anjs`HH^aCvBJn;K|q3z**Xb@lLa0f7)1&RDFOz9iy}<0UNi&6tE3!@Xd$
z`1oWqY-CT*xWHJ?gg_q_|Lped6&TA(KZshE8yeO=)I(P!vlTtsm-YH$tC?BSlc-<(
z0^=sq2ysU&b`;crA2DLk+`cSeD=fw{Y7cVa1o-Ig_yYA)@NG7%BFiCPT~4}yo4_A%
z`dBu#>(!|3&qi@|7c{*HodN>u&oc9~h@kz>$AzV(v7!kP0(6FE5{8<j`w=%?P_Ci6
z3DH8bC1y~ypP2^%?8y+w7!0QQi3xS8!ovG9*~yK?i4Mn-E7aSw(w!9*waTnDT9O^*
z7{vqzpQ9f=%YlPqa5upHT`F%&kuIjru;<BKFuFb9wya^$+65^j#aA?@|DciQ=jW8I
zMc-5eg#k`RdOD!^3w-(%f!<Hw`aTH6dFek3aLa#B@ic3JfzD0zgN2>EiPXgz=+N6w
zxByi2i|tlZWH8yTw}=OT>b@-tXJk5|g1kapgmI*Zxn8@uPj&Mpd_?-yN)N76?1d7-
z{Z57pET9+^ZIuuyb71F@3QOP}`F1rYHEyl^XS>?b@_iW*LzI;yN7A5E{eQ^{N$aDy
z_Xqi=&uh+;p$|`A$CIp*$U4fqHx2vbFp`6Fyz$dJ;0wpJ(Pb?%>^G~8{*dHyw<aqd
zVrJDIWcuBF?pJ&rNfu^S8f8wte@}?WylskaU*6ThJh4u&(K+ZaY}E%zzR_F6T{vb^
zkD4r36vyBWVg(s%$4c?*)+&+*Y6%wox>H?&udS=jI}mKY>Vg*Lg_qjz9~Ej$)YJ`_
z<1wP4$3W$q0oX}F(`9Th!O(M~@N6yZGkzL5A4sH@ic9tjKP?+>l9!0c<2TjOY!Ur9
z6{gDcD=s_De|W~qj44cJzy53Jt;b2(m=osA4pd!-ZT=Mh%37qYvlFs2LyTWRhukKi
zS`zPiP@ACGFJo~mK62o~tz;*~p?f%1JKF|Rzu{py8}=G*P+U^3GeX-)Z1jGWV_QZb
z)dVC!4}>+GU#lUi0h8<k@_|H7xCJmQGaak^O6jYr_~c#<F)hF&t=w4S*3Y*-VD}&V
zY&&+)gVYU^>{}U`Mh`FPNaauBOK7~xR*d<;7ruqgJ&E;VzkgFQZ(D(@O+;G_#VmPW
zt_nyA<v_bYo8UaVTf=(SVs5isDr_a`WmjZ1_7?Fv4GpJ>#L2e$_zSD*p)9W5CbWxl
z(XXsi8`z+ylTNf;7MC0{Gaf#Le0@8fez3SDG!cOmn*{lA?<Rs^<l=Q`NP|((yjSo<
z@nKJ(wCHLk{DZpx4{Q0`=6VAMl>%w3`Mh<Ts83x7c@6D*#>er$wVM5Ii8LVs|8CgC
za+7m{d<A;Xj7@FZgj3?9AaXZE3i$AryR0vF_t5Fd6VAhm=mfA=i$T{Me*Hnvuetcv
zVvZ%$w|r0?LfPZ)_-_4&1yo%CSj{QgM|{!;m$fY1`FKJsUT=O30WiPd3c5vXxa%I=
z@mP<_i{ZOo+}P_-c(_i$$pWIvc0<22J!dEJPGVMaY4>mLg-J($<6X=AAYRt;PT>l^
zJOQZ^MMZZVu8`Cja4-TjZouuq%ZRwDJ!_BHxH2T8m3Wzcdh6OK!t+BtXW8qW*d1yM
z1Is=Y51C&gY+L3$PA@YhW1;z~Si|S@DPKBYewChDWt}}NY0FkXxi<{T^d??A3wkd6
zwHyBQF6y;$Jbh{NyXy|M&sf%3Rcr6KZOj|A&(i)}Bi6hRN&Sr|$$Ka`B0jyIr8qfg
zaHRCl`@}pFA!BsHl5bXT$*0_avoYwkxuO|awiIoc$we}ccoS^n@nrL7YTm4{RA}1t
zM!V0<c_y*pW3RKNHj>XM{hPWiAM5QaDR(0#FnQ+LXpvw(=~Q=en^tU>N84POOnnL`
zwZv*X#?B=MFV|1N?B;4n(_t9i8VXqlk|F!rx!Mshu9qzXBe4~aG2Aw7?ur_}O|rFV
z(fYVdf-uHfud*Mwq6)}qG*IXAnT^>WcpbzCEm!4f|E#c19AmQV8#`$V{YTZ;$JNNo
zIevJE;A-8CNqhTIal%c)>&t~^xsnB5vmFv9?^_6|L5|3!i@%93Z`x#f=7=^*79S*h
zwy(aPT{IU)b1gr~(uv9PGvz2b*kRL?+==Fiz?E3@y{ey5*q=;HnX=h&kl!Aeq+Wt+
zKzPX?klmO+Km9(+N&a!23wt0!a+$#V`T9K<abkZI(A0OfWM(s}T=dy#h}9y%4`uX=
zdtg(D`j>E|cX0d%Vje8_oD_W3n1Ut(T<QNuyI(+6K8Fk#aoxZal$5MEbJcKcs=LK9
zgsqCG%I<^~vq3I_q>_g<d+t?;>o#rEx!<(qDs6|6sjEZ{lr*OR2~Oan;0g=?lMH)2
zg@FNrE*AQG38d#<3KWE13<>mr3#{Q|yrEPD6;XI+LF_w5QzsDs&1zZ>vWH_qxCs`&
zN2p_ar5^eVyHo&_8r^-NQjAKJq1Q0e@@<Jw0TP53#)&e8E58{P)Tg;runJ*b$Kg|P
ztEY~vyLt=-I|qz(#qRj<XmaVDp(qq-VQ?uQLNh+TVE`yn-~Ke0iW8^0k>0=u#7t2+
z*|#YR2o+GVb#jQXYx1V|?`GkyQ*mPhMF0o_m-(Id^MfX3kpctYS6!Exnm+1Mfy!VJ
zLRn<xD)qp8Nb_~>f;*SdCJYY()p}rnx?y-}fMl~lsm)!5IeLV(u^OOhR$u*lg1)g4
z@{U&GQb7rag|qm|>RRpi%hgP3*%(5=rGm7PQtEFv^Qj0KoE>XxwT<dVMnZyF{6n+s
zJ^8(c$7+HgLr+vphl|`W@MK95Tw|t4k2M5dILo&(0H*SjxA?&j8Ld^pXW|=yOs#4y
z##avV+RcVGf){4DL`Jz~lg1&Y?JeD??R=jf)kVy%8*M-hJyg3r+&K$Zc@?ZcQhHo;
zcT@FC(%7|cUVn^z`EL}Q=$Z#;hnMGx6?D<OSmN4@yoOtTZr{te($3)LKQ8tPNS<K!
zQ-1@i74NZd#N#YZC%Z+(FBa!u(dssDfgisP5U*$?%MwV+T-9F5o7}}Vn&@yFgKxbD
z?A<hBtxBV}lyIm?%nhpw7(NxHGPkX6V_?+OvUTzi<8aquA=k{C_=2aPL7pR#ryHGp
zsENwsHvJ1)(*qOA&a4a~@pPCWZFbMSAA)dkOG`G}W8LJ<SZAH7BtL|Ne8KnQ$nMjl
zFu0WpbM4vPQ__49q4=cI=TT&6d)vfqWNE~wzoN&Jc8o4a+KzpPZZ3zoBC=acGuxS@
zZeVflRPK*Rmg1B0FQ)<LWF>9uw}3Y_spfB-(u}*fmGO;s#TN$Y45-f8v5fPxzz;g5
z)x{@&50@_w8#1jA(GfnVbu_-na4%ivd6K|ShY~q)&+3R2(S@q?e)Rc~Tv+wyPmylG
zlkk?|U~7rW`q2C_j$EG4f#|~T(v3$MEchJdpXV;Y3#OQWp0A?v_^0?kn?B*KP_oey
z%BcxT?%SI!ZNHWU<uWzy9e#i#+>ZBDZ}!zAS3<+LBK+BGy(M-|PhWNUB3(YFWR>ud
zo7mh@5=8szH@FE=rM(|yOmIY3^U18s-8-$FJa)qQ@k0HvtJK}8XI>B&B#i=}#_>q(
z7dp)H*iPl-)sLJ-p{<S5F(11wiOLuUi{=jmpO|9eKxMu^be}Ynx9Zu1s|j9LJ3^LH
zJx}6}j%G|TcQo&2j{9bR(R|_G6=^lUq2UB4{^*wHXpG~3L}m8p-lnK^#JLL#<&91w
zRLnVRtY*f)PnBkj4l;zSc9|?$O2`ohKcqTvClyvPwE;HOmJ5rjH@4YVxA2E188@Cm
z!SX91fXSqIEMfmBv!%F!w(&ZP9}{NtQqi!lbxZN^tV4_yAKin?wnvaRI0~HtG;XvH
z4sU7sEuJG(RD0ADKPi#i_8#`z$_aqw|9^BNu!ngy*F!bZQ2`vWj=S|U=~3E#=?>(<
zWt?=`A#Z3xV&_0lE%N(fBA~&f$z%TTv4eU_r_RXZ7N;IHjd#Py%8=xWYaLKV7V|}8
zS@Cs|P0t4sG^46Lar=zFf-E)v@#f)lzN+T0(g6jp@qwx#q<eb<oc*?wAh{;SZ4qZR
zIo@T8X&OzT7%6I{n9v6=y=VJ9X>92_Z8Ojszi$#CjXSVw%pO+WwfgYKUDfWThYR!W
z-78)@+szJsU(|d%3z`>5VG0E34%OO~B(UEe^j-D7(s`F#wW9TE`P@c2hIILvmK|Dc
zq|JhJtZ=|A+tlg62WdObSsNhrchhArGP>O+LaB7chh1ZF(hsB1i4!M?!(FYkBn!D&
zFA}?|nR39feS{qEM}6+eiWoXY%nuAn8W!~jmek9V*c-|n6xS%|kxWL?X-vZv$Vfgd
zbGu*ONOL-wsg8Y^Q3*?`;(3R!M9${L8jM&mxy^an-12lnkIX!EV%ckoAvO6x2-hxq
zY1P3WonU`p;!@<svQ_ED(lr+>9li@^olpW&*qWhaR3|FSL$We!GKYndfco6<O24x^
zl|l*t179IS9mW2mNG~i+azRQZ=cR?##J2w+kHxxY^u3(<UQ5wB4Db&niq`iJy&K{t
z`5$m_${BY!R=MNdA!ApX{pZ7+c(&#y^C-v^mU_bz-aR)&)4b;nlUPuQ2anTTRmO>W
zbUs#0h8k3#ZN}R<j571@7vn`vWHE8$KP}3A$K)?JTR{v;7@d)s38c$wdXVEnN4@V$
ziNdr$$#GwHP$O}|;;Bd8WZ<1<C95tJi{4|wD1>}w$`%#Q5HIn2t~1*w=6W|yf+O<{
zxbj}NIZ517E!yHh`eHBfLyV_2a;_eMdwL}RXT@6~1t;G779yzA#C~GNBTJ=uaf_>$
z7%G&GbusSo%v=#5ISk^J^2)$j245NY3jQf-K|LW4(l1!~dEhgoDyge<hnw*kQFs-q
z${r08Ov(gSh^N7AdUx=VOJi|jRw{ECI4FnPN=1~4W(g#MxugjN0vT4IR5YO95I_%x
z|G~##b!TG0E3|cnYC0#Y8vz(QioTDHu(Yc97)@rcB5p)<TRgWcvXEB^P(dBq8P%DA
z-g%K*KePQdrd-|nOB7jhq1g<n$qZbS7y&Am0ls)cy`KFP55bi*cFn~t9@%I@84FoN
z*F_lg43$}^c8g?qi-7=NVJMPkHAVK&nl%ey{cJQxB{+-jv6vdPRFe>epfW7<aXIxg
z>pL8wMi>WNFnyn=&0EV+POTt@Tc!ro|1J3!B$WOw`O^Z#`Av6RbqLFoPrq@NuffTz
z@A|)c6<NIhw}ttIz$G5B10rt)fM5A+u5Mff%oyPwRx09NA#P{Y3^LdZ$7cCfaVx6p
za^7`wfS^hHY;}g8y(?!tEA70JP{`J5eLyn%sne4Eav_@~8&{R+zRc4l8{-GqgR53z
z`Tg%3uG`f$_~AVfOHYkl#_A&RL<5jH!wPS<QH>U8+HX}+9rmtYy&ZPAy&$I1YnO3{
zu2LG;df&qdeOVXE?1-GAuD+Bk9!-!;Ci+S&{~nY=8-0*rt*J}=vqN*3*;Rf0L#}MS
zZRi}})KMr)3ES)TCbZXP7mqre^c9_95}D&?H12oNKm9Tj;NKL(81GXco8zcIpLaSv
z_Uf2pY$%B+c_pYCIS^hJ1pV}>V5Wr(w_U7m(x<kY?n`iq4)3BQ7?nG$e#&-TxtCsn
zYiM)DW=HK~rR%3Owt-s(DY2TB{102M2ME4!0+*)0AuoEzX4S_vPyonE(Tt3GFO7kK
zb7)JE@gw@b(;fM(XB8nD{-@?YXTDOZe~wn`)0Fnumuz&kZJ==5Nrk2HNcijW#U-S&
z>z8`UFW9ixA$GXNu0@D!h-)lLk(YbnrcKw>p5i<ope&Iy94tZ4ze)VA#*oikd1K*9
z*o*G4FHe43S5P!_K7|<36)qIBYauiiu0$+U`${-n8MW1gYM9%0$n>#1>1_4*$^g&e
z*T`e`K<3h0rZ_5Q$#D0Z4VnaRnNJ>TJKE$o>?|cdD;87pcW*TM81dQk%%|s&EpFdU
zx{2R@x&`XbJNAA@k34VPaY}e)O_N3(dAgWJ+*(Q$<;-uCjiY5F>|&$G`eDud|GaB~
p2EI@^dY$^;PCNcr?|S_iv-OQHu-+Qa0LlOdWPmi$snNnl{}1Q{;tBu&

literal 2819
zcmZ{mdpHyNAIE1~o9o;)!%Qicj{D9p$zsYax1De(i-}HdEw{~`v15p&6iL~rlkhVn
zvHh$pPDrF&=E!A^+rmas)Lee+_uuc2^Ld`{=kt9&&*%9(@6YFXJ>Tb>hx2q+kVVJ>
z000G7ms4i|0ALCL09Ka*Niu806<v}5!MXc5?(OYKuD&ma(j`GA+U0B<03fIO<46JS
z-h%@GvJYHOVSN(Eg_9iO2Y(Idnp+T+myJS<Ic1|DclPt+5gpQB440mbkONV<Ves6$
znd?C8g5$F`Eajowc)f;pn~Nst0RbHI1?#%qrgF04Mbv4VYL^QKcQf1Dr`fg|S=4{9
zS@)_u$nlrlSOs)juA5pb?H&L;)s=BX3Zke~8g^<QU#kSmaa%(LduKW=M8?h!lrSC!
zRwfX4;yc$c(DOB}X`Z|_Og?nX1}3^Z&i}SPZB{T^$u=*nmAcX_PQJb0w<B$5%H9WF
z_q9KS{VT<!@JP+OecD-Ar~*60Tu=&45p!KXlAr8Zz^X5Dvse-6Kc9sjxf@K+>!K=a
zgO4ugA>t2Qd%k#+PnyA$uILcuR#~ATrrWDe+APGs7{7XS#nhpp*2A(wdc!>kF^aKi
zt+**~*~722=Yd}Z=-=+h2nPvrM!3&qwr>Myl~OCJ_LD~`ob-!>??FS}G^!rFz~F{8
zt_llH?+e*DsX2_?SOVQGNA5$DVRk{Z(mpfaUp7bfK0cf9&kV*w#jA9pkx<>*1P-J*
zhP<xJ4~VfyJajm{5&VGiFPF1H6J-MT=Y0v0(e9vmfl&A5w-~$Fzby-XL*sgHSnbX3
z=AWa!yCa=`X5H@JW=jcrfwDQUbM%Lex4$?7v3nl!4#Xs>^wXJ0QAsKPclp|5iLLuu
zN~ik`K~tPE>Ss}1L0!N5pnZ~E#6Vqt&W(t#CHy<m=jbEHXZ){!=Du??@A0s1u;h>I
z4g^0s{BoqBzNR@6^5He5K1<YRdRHaEiqHy7+?+z}%?9pS$`Xa#W&Om1uR1?75RRW0
z8X7Xv_rA|$?1q+rUto5tfrhIu{u0h)K<(0PpOdbfY2A()HZnfh4W8PY5+%0=Z#}*%
z)DKHb9r_7){xVq0Z7l$nY6#B$Bs(qhmi~roDEIunFNE{WWzS=diL=Lk8Us@s>ateN
z+KC}jCmwRshiV}w@X1H(lUi7Pl+2qW9j&WuGtRLs)?d3O-*8Wp!=UON@4{l23p>}|
zo~;%da0Tg0CD8k;({?ng#o6f$uSd_PU8iC<g5P7Af_hXJQMo5k=pSM}NQmZ?Dx?@%
zO`W}9zTQj~`aC!wJsqAfv}c4)YgTJxrO#sXmo90A4t8W|ASRT5(89$k%0a?#vs?{C
zyHly2Ulc?$71<bcrSU}t48+}!i>9O-A|+Wb^_5EOHONWym~)n&z%q(K|E!uRE05%d
ziPYrIIRiW!8IN`DEEB6N;Myu&C546W4Nt1e<vMey>|`BS1vq|&fYTk+hH0yjpz7>E
z=ivn1_jXL#qXIYb(R?Pi`peN8k&_8KGLi<`p^SZh(u8m@8M8h#3rx!+l6}jAh{uwP
zS?{0R5ECNriJe(J#AxC)vYku$a8HhbtO{qiHgSHD^KOBPHBR}C2Rg7EDZSz-)ees@
zo)x6X1NPZS>6?CslnMWdR<RGWF1jiZ29*^rwZzmt*@&un;oTrjOF2F*%b4P|_}=-s
ztKCh?gQum7g`yYQjJk8?w7`w!5tlz%>_Rxbe-gH<q%KlF8p|+)vH>gKjnHBA<BvVn
zq8NRRc@W`T_BiFyB?B|`<gu|0r`GiQ&O_XMeRx76C6#RXGZ-x(B==-dbFrtX+zrO~
zKFUTJDD_Fa;3Ga^bIR#2*X7z*s7GD(q1VC}%01?j*aKgozRna9X!&BPN)f*F@gHSP
zwefm<iuI*wZ{>O=Ai1aIuxcUT%7v)bvo`QS#6hz3O-%oh39FBXooy=x`5(x#pFie{
zMotqrs>dp+2a>{#A5JYj+clxD7vacdGFdH&CY{)HAA!?&_kzjQD(QPOazgIFRsXQ)
zA<#=Ej>_#XD;Tm<9=4}?pu>2`H<Pxo_(6tk>D4qmK~8~;V!ZLWYy+qCnTuIqj(o9S
z;x*>KbkJkAStajlFosdpUr{?f*GfUbeWAFP@OkXG^O9d!ku&D0mRno*g-fF*Ery+P
z=sO5kWkQ60w(S(()tD@4J4k3|Ue-)wG%>41t-j!Rm*<@DpHPiPt*Fq&<ulpK8Cjl_
zbxl&kw_1c66QZ<i@Zcog@9&~iI?z*rF9#G{8HWmnccMX(GR33N+%yHLR!xBgqh;TD
zwKlSS&KytEm%wNaM=`uNM>W7-hRdHVuHd6@tq>p4uvdWy)rF@^V3CVW7m6x>LfNpG
zpa`&W#ewW~6>9azoT(VJAw0B*TMTHBBGgiazz(5}V==k(4OLBhF0+DL4Y^h}=cVP|
zt&WJQ<eD;CE#}w;nUC<B`h&NYE2I&zliplBGv&aYw<Is9m82B;W|3`>^$ovy%ACGj
z0r^p2AHo!mUr#oH^zunUx+V3kEcqS35WhLn=}0ZN-i&KRQt_96i0Gu~A%+*Bv6%k}
z(#sow$)2|E8KGez*EMXJ-kZ6l`_D)K{sK1e$7y3fr^dg^3*O->$&VWGc2qM>wdI#!
zhrK3KlyJj3K7_#>VPiMX`7MdQiby47P<u#pQ!ZfQyXOgBn(1SfC=T0Cq91X%&+&;c
zk$1;{^<Wjt745qN3v2^w-s{+WK%9(v4a%6uX<A>DXsx2LL{{&nYlgIBe|yhJz*N?n
zn7NGFYYXO%9I7`Gf5<kBu`u?ohr`Bm0D1F|V^J0>anO-(nU_A@ns1^EkP+`X;~SHQ
zbXB?+Qkf~&zySayOhGI&1(4fA$`~`UIKbo7?7cKOV$I{Ufq(6~7)bh_qg#gGj5jY{
zu7Hi&@OF$TERmno8xozCT9hN1Mkbjlj`Gl7WWSxVT4X!_uAsA3D0fR>srKV3jH6)C
z@G7hV!~|^!Wah|QZpjlA6=-2NDt0)ua9NKkUjhW7A9ES%!#cKg5zQ0P#F_Sy8Zk@-
zuidjA!(S@0UE97QPqUA1TF<tf=K!n2loAW#Sex0lY#V&06apL=EG@>!)UBNyTXoIF
zasVgMxsl0zaW7Po_(?G?fdhl6Sc8<96+gX%tNZE58YKmYJx=Bg_I%<?ZR?(-5_srZ
z5_Z1=>}UA40b*MziwlrH7iQj+|7(EK&MgybI0c(WOePDt5D9nKRm7xjA(sG#s74}@
zPMV>EfLNyZU@yrFq`(rnGK_c_p^;YXq$+lRl%aw80bqC8as?KmX=~oY5@m8F=v(&u
zpF<u@#Ir=o!8%g@d&3*|BrB#7isB|g9Sjiz1a+LmZlL~TiFyygDFM2Ye$YjB4o1~~
zyf})P605J~a`<?ngW~|nOL2gtc-gRJfV5i0-O|y;GsP0$VRk-0AYi*t5mpe?oB2oc
zvuMU?UuS$_jICtf`=Fp<IH5X15%yFO{P*j|Oh^Fm!^Y_97QZ8L8UlS4;7jT4uce-U
zF$)qXuCqjys0&37iVyBBBPBDZ_^(m|wzPwoDWQ(wdEZq+q=zXF6I-4gH|a<Jty^i4
lT|FUTVUhn2W<9fIH{zv3yR>lly5tWFaCP)N#dIKK{TBtkM$!NP

diff --git a/doc/figures/compgraph2.png b/doc/figures/compgraph2.png
index f05f7eeb52189380876350d631919e7ed6038897..92309630483a9e1da3dc03441fa0b24f14dec0f1 100644
GIT binary patch
delta 2633
zcmV-P3byslBh?fkiBL{Q4GJ0x0000DNk~Le0006@0008z0RsR40PbZl-;p6He+l(T
zL_t(|+U=daZsSN2$D3}0eE~rZun7Y~Jm6k{mnjg2^C&vp3ka920%Jr&z<Ps$C%s^R
zdyxxh7?{mw(+v)Hob>~&2Ae1l#4|rcHPxid?y6aHz4ZsiBTMGb-OZ}%4|bOsBauiX
z5{X12kw_#Gi9{liNF)-8^v9Kze~1JLB@&Thp&tKnjB@enD@M7zsyrB|OZ@xD!G>@|
zL96H;Um~JLDvpRsLEhK8o9T;g^zllV2VyjH=!odaZ(SV{wj86Y-rzZSh8F%n(W$T7
zLbCWFp7}fx{=_pgG?a&jiZ{=Q@_SKRUDUGggVjzKHShZ%-=SG(zDKiAe@s-4SHT?3
z30HV1FHiAmsE0ZicUOA;e6El8^qkN=SK_CgFwR>le|P(>w6ZH%UmX>7eA~WRsd7U&
zp$J8&J`~RsR6WYlM!AuK5{4+pS5ToLDhL!5H$?e(sHsaRLj6%tzJrDWx$|Z5t>2*3
zAj1boQK$GeZbIJ_5c+}=f0gwI<S=^zb#*ai7mFF&#z}$NnRgS`r*O|E+h6#H3Y90u
zD1P()U(EZmzVWKI$~VR+!Ee0BFY8aDM(qKpkAm64`Xt^l_PQ9Hnxm{5#W7Gk{;ozv
zaMVW$wFRM?3Kpmo+E&*G)ZW}g#r1cOD1QATuZ&@1j^gVZ_Yrk>e<R#C1u6kJ&JW@P
z^Q;Gc>lKbOgACR7V#~txt+n&uqdpsN0NH{16Z6<VCu}33{>)|<^JTHTrib!&@y)-#
zz5XDf;>~{Y<-hhqyHwM^{jj^ccjQuiz6pO7PIMOKb+py;?&13G;Zj0b58_sccdyQ(
z!c;|V?jF|3Ln5L0f4#VM`K^9GnB@g3lTqvZ5%oUNMOj(O*U4x34vIB%U4E15qIg=M
zvYmwTc!64Pv{6ZkdM%;c5|wOqQ6kCyvQB@VNT>-<Zax0}QWxc2ZvK9q{_IMq**AB$
zf`32NMY)&T`|IppMoqtYaK+luMopa6eez}fPC_mJn!Eg?e;etd!d>iTak)HnK&>{r
zIN25J(lmD_6rp|(s09FpxLO;we<%Z>nzea#X+G6jkUx9j7*LE*gd)_lqMT8p>`|h~
zt_Ve_XGYEOP|-_AkzEmrP$sAZ4P_ysQoIVLV{EIhXbkH9JLBz(<3z>ka)Rkr&vBwc
zwP9A<mxS`ve});n8`=J9D50Xlp{NsQ6>7uOkNba9)G1j-nQl$zYQuQxv!g5(WqM@N
zQX9tfAm=DiBF|5)A*@$(b=52bA8%>#mO;6~M-LS3Ftd8RrJ+HXDWh>L-+aDn-hW!s
z3Xj+*zEeF19_Tv~s^0DRTKe>D_1{((wNpIh;774$f9yMKO_H3aK8maDJdG`0HE&J&
zUGuCa%8TQfk)<ALsTA+%MiV{MMj4<UPaNqT<&#b-9iU#<hdQm!*i@~SmP<?L3xX3Q
ztDis(RiIL(>!&&>>4$|{t3U^}P^K&LRKwOmWy<JXK2bkt^zL`ecIJvP4LToJqgsdh
z7pM1hf7rpmpu?W0YEx&!1D36pv~`wjZe*bce9M-i;V4JtMm=UXu=VBI&U-%}Z0v`7
zkoEZ0Q;OvH&Lu;wEy~eF)n@@JBTMDr;|@nv4O5>5C`(cor|unXT~xh!SVnO}RDBkp
zbdxqp=mv(?M(Y+zV`87GD5i~ig@^L-Pyrswf5AgBLK&g7&jR$Lw9f*Zi44B#&f0^u
z3RRy4kb{gc3^H{<ZiT)~7uD*i<bd4OPwJqUf*RIM_)6!+)y``eD!}Vf9i_{3)uF6L
zNAIZLM25o;TaAW(&rtQr834jijdctlBh{2NW+s+87PWQ9qFui>Q-Xn3DyvK>f_%Bw
ze=KgH6hW?(tAj$E<4n|-%Z}Ob6S_QUjhR}MjgHeujl57AXs|8V?df~->Np1;ydGw^
zn}e&116num*h11oexrOgOrCn(Ot;lf*Ag}gT{~OKR#_H0EFOir>vOSs-R^YlXlP@V
zWw7Mc`#3qS-08^LAAtvZ+8%*=HQW@>f4Ew26<cG4!9yXQZ$;j&jF)GjQFe?l&-d+K
z%2}yv!RL$9u=Xe2aG?80NDjR|Gn_K3F~nhp{d=vX4;3+hY-ZTMKZTk@P@EYK6`mM8
zz;Y9|1`n_dQLWAem~1NTZ8&XZc&K@=WoTc&0HUI9UvXflDG=2YS9};Mz(YA$e=(*#
zY5}5btQgZCS7$I3$BHqnfMu35@3Jefn8g^ot0-nkdPP?QacE;_W|O4z0mp`-7H0FU
z^8v?!qI|Qd+j*1^MLB>WQ7<ZjqHOSxXrZD^!_`zZeA96CN=2p6wrWC)g@{U^sHR>B
zImI)aA(3nYHXOCULq*^i!%vMBe}GC(VvBi<X)H2;hpXhIViCOAT|AVHGbAc|(yA-n
zAfR})ma<C`P<gXl0gf^VsM?T3je-mxgTo?7h1$Xms@~WPO6z(lqoB!VFN!ZQ3QAOM
zRiR+nbjvMeUX6k!F)sHBh#Cc(OOzY=c#VQtdz>{83zz|SwP{?&0;c$_e^9Hkn!v=v
zm9vyvDmgH-a%D?Z<8OiCrCPBqQ82TrGc%Wq+dKoJ%G*{_fGNpEwUM@bC;|LSGLNPD
zUn?sEpy^K%N<dKj<YLGyC&oi@Jk-f-3xx6sMW~aDuh3A3?gjzH-{Ya`z=wdk;Akit
z55-1`!ZT5UhqCcd4j#(Ke?vv1M9uL~=^MOwm7)b1Je0`se31G<LX8_0;GwQYS*oB&
ze~X5i<DoJf6#tL7+u@+xtKc&dDws`|NNwc@izyOn8Z2f=s3=$jc%=$nbg2+gGk@vN
z(M)vu4;)mu4;EKADDjh*EvfS&6rl)pP>8i!i;BXi8KDS8C_)j6e^7)X6rl)3C_)j6
zP%j_l0rygl9wmS~JV%e>7`v>;jDqfl{W>UyiDF|!t?*E9M~Qlkhnl@`)RTXLZLo^f
z-yI66+;)!>X3F$g0%Iwf?a_Rv-i|I1YFKARv=C}|h37m?-db=ZZT84>aAS4k-Z9lN
zj}rK_?6Lz2Y<2i4e{AqEbJle|^k6Dtq5#B92#Bd35OY~9dMaz6tmhUONHzeDWuf3C
zFi9U*-`IO=3o>?~x!bMyK8K?YX8}NYiYfy?oCSywQ1w{=1k~Xyz!V-B=q!K+ip~NI
zN6}e;!R~xG3ou0pGBpYz$Q;fByuie90uP1QrE<KE9x(@9e|t~_Lm>{;pxNGf2H3!{
zsNXa229Dc-vp5&a1#y$1rCl$fh6Oxnj+&LLBLyyeabN*Ob^I+M<1mw9wfb?P0lds`
zOwEh1=p?Ed@(hlu)BMKprmAyUSSYKR<zFp}GYk}4<qVj&<rot39PL34;v*pvi9{li
rNF)-8L?V$$A0!fqL?V$$zaRY{<C}eFy7v5u00000NkvXXu0mjf-q-MV

literal 4557
zcmbtY^;^^59{+5lbhm*tNOuUMLrKTTNeDQ=At4ef3~|JyBm_aEWo&>n62c?}NoB<7
zl8_P*r7z#-x%d77_dd`2oOhpdemJl9`+4567G{R@G(0o_0MHv7=~)2)<uL$&hCpN&
zmRfebl?z5`aT}_Cetv!-TDN@<yI_}mjqCydfR^>21p+ePasdD{(pXO$8cf>EZcUgR
zXYcB1AoBw=Es=Y&L6|R-8}(nKr*&D9a=)vsjj17tgI#U;=$s!``jXfT-tjXgJx>9R
z_O~T6`37AfI2I0rpL6{>FKUtKXO>K^d)lI{bkFba+KJTQr$3I#`3mu!u9dboI9~p5
zY?Lz>aa=tlJegDJ@i3guiTwJIu4nswF^X>6Y6XFbxt~2zOkcEEYqU=x%uw~393_uH
zjZCb@k3Dl1pLZftGPI-S&xzF<j)hZ;`!d{}Q8zP}jW-X>-K^iRuiHW0+pPI4%`|%f
z#R9Vy!yLk@{41~&=qs5%&t1GOe#lz$Mj+mQ980R-P{D5wzTAvZYso%PGTz&odb(;}
z?fq(T&0dmq`cY(BU@dTzTLj+<7)T-6_^kV!eVH+KT80;|MjR%+@|!|@-&lU6t2q%O
zmV1Xmjw<uvVAR&mH(8&c8$DDloNKoJ0oU4>b@3;WGDx=I23EZSF5cwx`1Qyz?d&g5
zMxLtPDQl$FK%`ow$6wp!?W&n8@BN5Gu&0~(-I*Wx=lg|lzP+1SZ1F6$v9B3C^zYwl
z%uFf9wmA3-!@u*xM(kFm<QbpqNO~!22Q(K>C{yGfLEv8sp6Is}vs~WB9~ng-Jfe9d
znY|?kX}UdpYUpTfxh)n;xh`hqhj7qlTZ=143pp!Nhy>EHy`6M1XZOF0rReQ3!!nc?
z={zW*@=5n|QI#9{X#%&l<jv#QaFQc%RZahVo<u#7ceCMk*<1Q5evi^;Ny12df@ehI
zLpEYsJ36$g+8R~Bk*q-Gm_{W&vw4PzqdSo1zTSDIw`aiO_hc69U|8pRIdGe2Clgpr
zFIZI!siOTevGI5?<$!>ciaJbosSIDxtZ@67mk=`<^vk+rNXdD)fFpV|h;-JaOcYq4
zCAQtP8p$8|RqXi&$>vRpjOb&3MZDf97#|bB?m8r#ySZYxbLm~@_mW($obJ?nSXQcW
zkCdm6VfphjX9?_O%901fLrK7cLIkN@7c5?%T78&7K2E~OfNqNqd!SNh3N@kzMZ0$d
z@5}oW1nx(jrr>a^HN3)Wu>k)0agWPh$z7wX%+c^{(HjnHdY_UqH#1!^(EavA&8WI?
zN5?BI`_1z)Dr<tq$X0ZX#pp;bv-^^&9phAig}TV4Zz^iVeRQTz^73=IuA-N;n6Z-&
zGKTkaU8Lj8CTV|OGLaCDDm!R<<LY`k5|v?Y{JM)~;Id49-^Rj=0MV95A#JrcJ-38R
z*>r0E3fT^K%?hWyIAV8g_@S(EY2MQ7W!GVzt?(i8@!d}Qzc}L7?};f#B9!c%Knrzj
zzy&tY>-Bc5wdx#i3fa{?EekSoKJb<&X2Q+xNO^2VHPKiRW%8gV3wnonEwX5(5>4;%
z5Yun@fAq3j)_RT-OT?QlYaA%1&Q?rn{TyF^F-P{TF>JKbdT~U^x~FD7-x}Y=F1~O2
z_4AJ>vB>7@ww@w2UgEr^abgCw5jJ717Y0EI?J6&v*iy8uP8gT$R()4Hx+Dq)F5|1Q
zzw)fe+lXq!2!+A2!Tj!{X6{1z5>+~|*N7XY-7s^+b|*<lA?J!=F!{WZieEj<+;h9L
zKR6yqXF0(|RM;9_7wbRh-}^;58H`5KmDIg}dXM3LX(Ymw3=})Dt}!?`{s&6dptcKp
zJx+Xe1m`D)>A&y1ry4By+E<WN>s4BQY;_0!qsac8Ot2pplAu)TP%(}eTi;2Zw@8Ru
zWjJ)1!~Q63tW|p(9bM-#q1^eSU4Vb|vcu)%07U+3==Apyn&Ti-OQppxG*#S&u+h)%
zWy5$*yfo90^-1*%zZE{zRq(tdr(fa1DnaQ6`417K$tpgMOW3V(5$O$T+py)I#hq5a
zzz#cQdT=VOl7{SgX|r@g{-1jx1PyXkoFcf94MI!mg+RWNNk~7SL@<2r)g}Z{s~8Ye
z!BNhNR)8y1pu+!tKVXOu0xVi1n;UDGnD)G4kjtQ4F@hK3Iq|^^orOygFv9E$0X1Kw
z0eAW5j6kbOWc#8hPHHTF?wNE&<~3(5b17R0dni3f5}f-UQ1F9vX?v{e$|V71F6SuS
zibe?G<fww~Zv+_81Se|~wRg*L3oOa-b7qqqSh!+FNp^n)YYmA&@$yYFTNo0aSk(#4
zHNzo&5>!-f<Ri}coMA?w4p2xXg4Gv=qO^re04r>eje$st1RJ~($do1*4?N=6H%9#+
z|A`ODLfKHqRPQ|skb`t;U==9}Z}02t`_m8r5ZvThkz%R7agjYzVVoA*PzF|VG)=-F
zHxK38HcG*7BiJulSiW2bvG2$ouIv!}jX_+xP%@Ue8w7i{mCxK17D=4V9=n5Uc@!Va
zMYZDU*$6oVljB|yK-|mHL7hQn%1K_uB8-~CSr6psDyPB&vXC7%e+PDS3?_9x&DU2J
zn>(5^Q(N24j#o<1jH#a#>f?|(7^Jgkt~)Q8(3<&FSH%`z5I{D?;uZDrc=QWs>^>zT
z#wJRM>FAhzN4P;P%m;N;ufxlA^yF862$-P`_1z}w3=Up>x5j)MnB_#_1I&iWWmL2X
zo7e75aiDlNHvlL4x`I^>nJnS_05Y~ou`t@;bCOJs2<mJi&1<{vcWaqejPheEoaw+e
zc(u#c(`?7eX>uhL9W)dGJq1W4t**6+iC!;%SqH(fH$~X|oav~?lH$xKqdOwc>|TL3
zfA9ZpJr6yGRb?7v{YpH)+*gEMh$ha?NQQAkEn~;e&e}h;weGv?Q#S9O?fX1CcQ%0y
zwuEn<3>rzvr&>;b;*o6SZLorJj_%;cv$gi!izsW@#MBtCJlwu2ErfS_s~GpNya-G8
zeb?aLpN3Q$fic>Zm&YxoBPcTzYWQ@q?0oeWY~gdTJk!IaSD%%-XJkHY=f!y~NVVC8
z&gx3T_V@B#PJ&moqN<4SlP;F<KSkIZPZmQ)<N4B*;pn{j^{*H@ELFtEz1H+@30=f%
zp}SvoXTOZpS*rR&18gP@e&<U|$mJT9wbi=07sG;Hu@MF1dtVz+N}3d89=PjM?cc%2
zy15s@TKLk^4TP4kZ3^N||77H$TN`M@xNgA&=36T}$7scH4T1BfrB2b-(9k(J`qdxO
zb`ORQ;bairLLXKki0_f!n&n#XHphq`EfRR`1`Dwrr*2#WpVV7LE=~(<2>wWaF5nQN
z6<!n9B%zAiVVn)VSIt4(9H-|j=d&(`xe+QGrWkC3{QE2ZPIs|<SlZQPoGsJv7EQm}
zm0i85Tpis~`UN!*>J|Ncza5Sy%GsNr2KuKvb6cM<A@x$HYM=(}lZ+DT<NNM`eH26T
z_ENIf$mIS@b1Q&~mO7z2oaOnQ^MaiXfKe1>gBV;0I<9y9Z*BTtg_9AGR{F#D$P@H1
z-QNbXuuLbxr^(XD`-DB<kxXs+(5*)sLX=^W{$nt@jAbf&(_xwd7B1@*m;&VXrdZf-
zg##AmKMy-mo>UMCs<PLhgQBxC1S*4M7GtR`J*<4{AnQy5X{JBSt<M6xdQE=h%34VH
zc~-_Hvk4&FbJHT#TUKW=lrbepXKC)mgqIhdom+r{s|tFq!-7$IaXQpCah^?Q1tyrr
z7?vd79;_U-#800!%`Vn4RLfpAGNZ=(TluX|cJ>jk@W59RFUIuSZrBG{4Ozz44MxI*
z4P>l$cK`UDKC;RrsIow_3~^KDd86o)*8ycx_BydeAM!fZr!bFOu0JW}T`NkTI5*x3
zJ)2HVZhCd<b9`zyybs3Z=D)~ry;K?Hr6+}LtW)!1NTn%ot$RgNAUkSHi>*|v&kxkT
zI@N7UCzZ=}qu8ZbOCq_;7Jg$cUG4?0QB)U;ds$wFuqRAHw>7$0JVBE<t!)zK{VwXK
zes7FiP@Mm|`1w=liJX{I<{x)`ho6)vuf;BLa$X%#<;OmOQ(Og}SKdRYJ#)hQv5xBw
z7oZn>*4~BawlaCsTV=#;iFUp~_cv<0qSH^=2^JKhfiJETF78V{vvsD-80zYrX0qbt
z4AOms)K3|aX=Frd5d6HUk&aK>AB@2i&_hrCd1)WqWBuN-*^Pn4QvRc_^o!L00@RNx
zu1b&;R&^Wgzev|tMj$oTyD>`i3n=+Ex_7!MBQxz6dZB&z<zBecKqB{&q2w*fQ`^hx
zgPhqdN?$*YyMhE?t#)GsSv}!>iNLO|v*FehrjY-~PrAA>!!m9`%eqhAcuEwdeY^}#
z)qs89ML$I%XP(81WzC9a#rToHrt9c{FNGBgJkmO><xX9%jYhJyUrwA`T23U+?y!5*
z>rL9S$>nQFhF-TY;*6hD^6hUu>nT%w!-(P-mA?orPH}S&ab8BNP+x{UUg~!tf_4S{
z3mK9`hSghp*B?;HzIOC-FMN=egG(zjAeE;vN2aNQC4wG5kj>nb(UAb5j^Gu%#3D3`
z*_OVtOaoA$u%$Px$n4JoiI=jW)W8g)s4;34DuM(#pBONG4^+MJtIw-~NfBaMbfV9I
zFo|})_PeRi<ebyilz_|FVsuyz5pqD!RDO+Z-nLIpqeOX>b*1GH9-3G#f92sUeScP-
zCPmgrondFETd~gPOKl0i<{Wk}^FJ;7h>1v^-`IhTV*>7($SocBz%217Fi4pxE%xFu
zGbvS<K`WbJ*So4rL?%!zV$9?(KZOh!-<fYp0D*a!h_zr}Gf>BthV5K%trzQ1A<#$z
z;j~I6N0<?M-_&0UMx(&_nLcKewpmoN+zDU>G1OQw86DG5=T@BMvf{unN|z<~7HDJ&
z25R>}VcaYL&2WQ^Y-^N!#dS~ulLcb}MX7W;ShSA*l0a*v7xZ^VgroZ462u*E2f$n^
zysaIO#Ft1_@vG(%R=^&`2gmvqo7E+v9v%MDqGY-7(7H&)x0dHc?v+h$?(^cr(M-wo
z-ORkYIkYv-0x)1B2uY>$#RH<g%ZjQzL!Ec*L2tjY1i$O;@K`6|K)D3@bYX*KSE%+?
zm_i7G-4&{-=k5Z9JN$}u(m<;v4W@swq+m*muna^J=SVMNr@wVFFwIe*!%B^K=H~g@
zdVU`%egLfcFTQJGbt_=cOLWwbzCy5-*tywv5rbQ%ddK-8C+oHHVD4a$Phg+F>#V(=
z<{<P|zP;9;qe`_OBae}i3%Z=OzOKJ227{3L-}!h9w)Q#HLRBK7E-(Sda|c@(X0QeX
zr!0JGs>|<bbGq5Vm=|9JdFiBHulD%*eno_i(bBHeD4?@CwLTu}%Rp88)SHo4@I~C(
z6h?_TYR4f^%kS{toA>TTQ_SS>y&(}qd-Eps2Q>ihFbticaGD;y^0{u^;1Bg#GikJz
z-Bf8y88o<CFaBazxk*ccB+5z%Q+i?1Xv3cxuC+FZd4J*~w5AL1Ofy{DbLliZE==58
zQP%9qNn5`tPhwm3QB0VCR4eL3!{a6Mra+|r+jZTLKYgR=*L(J@5(HoR3K@P&WJe0$
zmJTLr@?_oYDLS~Fx^X*b^Um;^u8UVr)#el?qkvz-Pqab4{H!ByDRNHToX<p)x@B)C
zzq~AF5y7{qi5p0WoHity-V27pn`fRu;TLU1_3vA>kVXb1nu=AB@!t3|8SOFHNYB~S
kRrRj_XMn+hcR#(lY|%)FBSs5eOk)6JeKWms9p~r&0yU?F?*IS*

diff --git a/doc/figures/compgraph3.png b/doc/figures/compgraph3.png
deleted file mode 100644
index 92309630483a9e1da3dc03441fa0b24f14dec0f1..0000000000000000000000000000000000000000
GIT binary patch
literal 0
HcmV?d00001

literal 2645
zcmV-b3aa&qP)<h;3K|Lk000e1NJLTq00K(@00QL!0{{R3?qx9F00006P)t-s|Ns90
z0033O(|!N|010qNS#tmY3ljhU3ljkVnw%H_015R;L_t(|+U=daZsSN2$D3}0eE~rZ
zun7Y~Jm6k{mnjg2^C&vp3ka920%Jr&z<Ps$C%s^Rdyxxh7?{mw(+v)Hob>~&2Ae1l
z#4|rcHPxid?y6aHz4ZsiBTMGb-OZ}%4|bOsBauiX5{X12kw_#Gi9{liNF)-8^v9Kz
zhy)2G5|Lt|9{+KSa`EabM!CGIJQ%1;{QJnkhHykdtLPnHBBDkrj)+P@-q*UD>5Fdk
z@k*EnVl;E;i0H|0T^$m(9HXn=;5m4P7XCodsju5YviKpM`8*N+#4|HAl!u3kH_wRj
zdr@0m)Uxk`)lL^R@B1L%p;>6YN3&2&RE}4{9L)(=cqlJV@oK1tIv00Wdj5Q_kN5PP
z&^=e;r=2j)TPlBd`>nLHD_LJ16?J^uzFDbqLpY%bMW{X$&lFTW%F;%;k%AJ2D92Y&
zp&=>=6cjf^`FN<QODID9QBb~vh61_sW$~@wpwuA42S-t-_%?1r-xLt~f)bVW2jnn&
z0(EsUWfzMX+r~+O+L?C~)~9gKCfi^5hYFP^#wdRC{$I@dvcB=Ew#qlgD8X;M$1m$o
zqDJijsE>l#!ullMG4{F`oSLJo8pSbCJpQgmMR3$d3AF{GnhF-E6xvqT2h`r&M8)-Y
zk0^fqBd?5MV~*nM8}|`)cO%?41u6kJ&JW@P^Q;Gc>lKbOgACR7V#~txt+n&uqdpsN
z0NH{16Z6<VCu}33{>)|<^JTHTrib!&@y)-#z5XDf;>~{Y<-hhqyHwM^{jj^ccjQui
zz6pO7PIMOKb+py;?&13G;Zj0b58_sccdyQ(!c;|V?jF|3Ln5L0y|{Jxt$sh4<pnB}
zQS1B>^*+%>Sy{^0$!GZviZydxev|5=cv_&corLmufm(00QAvq<Euq{Jm27oUBFX-;
zPJf<As0mMQJ^uYt7v)`U{(hbQ>`JKFH+Q#!e?QelxtH7f>+D`eO}}|?#oEzEO`O$z
z@@4%_LM{KAyZoaY>7v42>}7GeJaj;<HoG|473<P8cP12}eh;Vx0EM_(8@7Ka1E89<
zd39+%)mo50d*K*Rj8KFk)U%?TQKIZoqR6fYMW|;+&GAstOGlAi5sFYIs00mVA)->e
z3Z`RhtFLGb>i#?9?Th0?#p-f`=~mBiqC&M{R@;|^^3;YIyc^m6YAB(i!l9@WXBBG0
z)Q|grQ`9M0MVW3*=W4@v>9eCO6=iy4(o!47^dRRbQ6kSzts$&eb9L1$10Qc`@s>fk
z!bcAj?J%=?yrrQ*m?@)iEZ=;-Yu<la(h85*D85rY2Oj7<5~|+q_*(k(ZS~()7qwG7
z<={uLX6!p`O_H3aK8maDJdG`0HE&J&UGuCa%8TQfk)<ALsTA+%MiV{MMj4<UPaNqT
z<&#b-9iU#<hdQm!*i@~SmP<?L3xX3QtDis(RiIL(>!&&>>4$|{t3U^}P^K&LRKwOm
zWy<JXK2bkt^zL`ecIJvP4LToJqgsdh7pM1h*ulV{!=9&VQ)j~imaUewb(U;yWT6Lq
z%a)?yC`aW+J!Uts_2t^mdp{p+?1y`h_4w6OisbmtB}1((%F#vDX8|fBOXc9>4o6iD
zQ=bJWOHvo7?j3DiRK0mvMsY(_eHNf}lQv4|28Px~>lR95VxOugrj2@qhw|}I0UpZ1
zLoq@bp|sBe^rN)T0-T8qzUt1}gS84(p9PSEj4%u`bwF-~zDyU@>Z#;_+|^I&pqPRh
z)=l_I=f%~|YZxlP>rx%1%XHPDtVT!gsNY0}!w*}HhJMdb^~o6k!cdKM3?L)blr&~0
zmO2)-b;qJzzcy2XfmSN3OeunVxz;Rhp%g)`l&gb6oa0Q?m&=aX@DsW`X^oj$l#Pzl
zNR7Nu8fdUB*zM_i^XfPU9=slAx0{2jivwCW@Yq7qM1G@uHcXy+-AuRDPuCJQ3SB!}
z%2rtxIxHTAy6bbXdfo1H?PzFYm1VHx)%!R(uH5O!+8==jd)gj>dNte>&$wD|6<cG4
z!9yXQZ$;j&jF)GjQFe?l&-d+K%2}yv!RL$9u=Xe2aG?80NDjR|Gn_K3F~nhp{d=vX
z4;3+hY-ZTMKZTk@P@EYK6`mM8z;Y9|1`n_dQLWAem~1NTZ8&XZc&K@=WoTc&0HUI9
zUvXflDG=2YS9};Mz(YA$F{V9g0itZI7}FkCXD}4UiZQK#WtKDVvMaEd#TdJ*C}v4|
zMOOoHXk%w)lce(j$A+R7X7jD{0mp%&e6y+Bd6W-DIe;NiFDinfZ19k1p`uK~)l@Zn
z({S}lMWxWTYC?;Jh)STSrd|j+#WS2Ck!%At9JRnhMc^32PmL9TN={;nd5mc+GJuDx
z<fLK|yxCnml#MeaDtpqZE8QTVc(s<YOA%0cvs?j=G6<;JkVK7w3?GBTB1nbW!VRk4
z*bGYRdMTrz$!0H#FEI*ARBctEVA*ubEoEMff+aC7_X&s^1)EEh8~J#Rf?0c<H4qD!
z0e7`&T*d;X_^nW@vYNoe!<Dm?TPis)vvOriRpW1g;iX!!Em1JDsxvc}i`zT{qRQJ=
zQh+JRMYWN(d?*3@OEQn8`d=$61EA?o5=uZ&{N!TDEGNc8aXi$?Yzu_)2}P)ri?7g7
zhwcUe#oyzh>cEG9y5MLi8xO@sio!Edfrqm3P!1l-$3sP<M9uL~=^MOwm7)b1Je0`s
ze31G<LX8_0;GwQYS*oB&e~X5i<DoJf6#tL7+u@+xtKc&dDws`|NNwc@izyOn8Z2f=
zs3=$jc%=$nbg2+gGk@vN(M)vu4;)mu4;EKADDjh*EvfS&6rl)pP>8i!i;BXi8KDS8
zC_)j6P=q2Bp$J7NLJ^8kFCXOr_fn1?C4f6TM~~tdyR65Ig6@X>Iw*#TVq-+D@KA3@
ziF%EPn!Rw;lYfG3u!_~+9SW)3c8?Qg%Jf+RV=0>L(R`=gjxG>tSZ7AG5NddZ=R8f`
zT5u$7_Q-Q^V|C=-G1W1T68N<2vI7cib@(Z4@G*1Nbv^W8Dq^Al#7qc?sU8q>SuA=g
zYoM&>78po20FGs$;3P0fA6MVldut0acA&Z2t@u8NqYh^QKzWKP13#Pvh!9ZqSpWpo
z;Vi%u9vJ8>fCh@r0t`pdS%AUrd^ig*MF=uA3L(fG&H}u^#Bl--h1jKXypA3*2VHwm
z1VbSX)u7qldIs3Qv8dlO@CJ_CfwMRl%LQ?hp`~3fp@s!KX^xtet0M(2d~sj_MRoiw
zA>%NUVYT{kp#i+ia7@jMu;?VJ8uAQ|s?+?&@TRJBSy(8mndM(Ci!%%qTjdOxx8)cT
z@*M3!4&oyr5{X12kw_#Gi9{liNF)-8L?V$$zaRY{<C}eFy7v5u00000NkvXXu0mjf
D!N~1T

diff --git a/doc/install-on-macosx.md b/doc/install-on-macosx.md
index c7c30d464..3c30c1e46 100644
--- a/doc/install-on-macosx.md
+++ b/doc/install-on-macosx.md
@@ -1,14 +1,14 @@
-## Installation on Mac OS X
+# Installation on Mac OS X
 
 Ray must currently be built from source.
 
-### Clone the Ray repository
+## Clone the Ray repository
 
 ```
 git clone https://github.com/amplab/ray.git
 ```
 
-### Dependencies
+## Dependencies
 
 First install the dependencies using brew. We currently do not support Python 3.
 If you have trouble installing the Python packages, you may find it easier to
@@ -22,7 +22,7 @@ sudo pip install ipython --user
 sudo pip install numpy typing funcsigs subprocess32 protobuf==3.0.0a2 colorama graphviz cloudpickle --ignore-installed six
 ```
 
-### Build
+## Build
 
 Then run the setup scripts.
 
@@ -37,7 +37,7 @@ For convenience, you may also want to add the line `source
 "$RAY_ROOT/setup-env.sh"` to the bottom of your `~/.bashrc` file manually, where
 `$RAY_ROOT` is the Ray directory (e.g., `/home/ubuntu/ray`).
 
-### Test if the installation succeeded
+## Test if the installation succeeded
 
 To test if the installation was successful, try running some tests.
 
diff --git a/doc/install-on-ubuntu.md b/doc/install-on-ubuntu.md
index ec87fdf7f..47a7f05f3 100644
--- a/doc/install-on-ubuntu.md
+++ b/doc/install-on-ubuntu.md
@@ -1,14 +1,14 @@
-## Installation on Ubuntu
+# Installation on Ubuntu
 
 Ray must currently be built from source.
 
-### Clone the Ray repository
+## Clone the Ray repository
 
 ```
 git clone https://github.com/amplab/ray.git
 ```
 
-### Dependencies
+## Dependencies
 
 First install the dependencies. We currently do not support Python 3.
 
@@ -18,7 +18,7 @@ sudo apt-get install -y git cmake build-essential autoconf curl libtool python-d
 sudo pip install ipython typing funcsigs subprocess32 protobuf==3.0.0a2 colorama graphviz cloudpickle
 ```
 
-### Build
+## Build
 
 Then run the setup scripts.
 
@@ -33,7 +33,7 @@ For convenience, you may also want to add the line `source
 "$RAY_ROOT/setup-env.sh"` to the bottom of your `~/.bashrc` file manually, where
 `$RAY_ROOT` is the Ray directory (e.g., `/home/ubuntu/ray`).
 
-### Test if the installation succeeded
+## Test if the installation succeeded
 
 To test if the installation was successful, try running some tests.
 
diff --git a/doc/install-on-windows.md b/doc/install-on-windows.md
index a60403378..a240a128d 100644
--- a/doc/install-on-windows.md
+++ b/doc/install-on-windows.md
@@ -1,4 +1,4 @@
-## Installation on Windows
+# Installation on Windows
 
 Ray currently does not run on Windows. However, it can be compiled with the
 following instructions.
@@ -16,7 +16,7 @@ re-running the batch file.
 3. `git clone https://github.com/amplab/ray.git`
 4. `ray\thirdparty\download_thirdparty.bat`
 
-### Test if the installation succeeded
+## Test if the installation succeeded
 
 To test if the installation was successful, try running some tests.
 
diff --git a/doc/remote-functions.md b/doc/remote-functions.md
new file mode 100644
index 000000000..7f28df99a
--- /dev/null
+++ b/doc/remote-functions.md
@@ -0,0 +1,79 @@
+# Remote Functions
+
+This document elaborates a bit on what can and cannot be done with remote
+functions. Remote functions are written like regular Python functions, but with
+the `@ray.remote` decorator on top.
+
+```python
+@ray.remote([int], [int])
+def increment(n):
+  return n + 1
+```
+
+## Invocation and Execution
+
+Normally, when a Python functions is invoked, it executes immediately and
+returns once the execution has completed. For remote functions, we distinguish
+between the invocation of the function (that is, when the user calls the
+function) and the execution of the function (that is, when the function's
+implementation is run).
+
+**Invocation:** To invoke a remote function, the user calls the remote function.
+
+```python
+ref = increment(1)
+```
+
+This line sends a message to the scheduler asking it to schedule the task of
+executing the function `increment` with the argument `1`. It then returns an
+object reference for the eventual output of the task. This takes place almost
+instantaneously and does not wait for the actual task to be executed.
+
+When calling a remote function, the return value always consists of one or more
+object references. If you want the actual value, call `ray.get(ref)`, which will
+wait for the task to execute and will return the resulting value.
+
+**Execution:** Eventually, the scheduler will schedule the task on a worker. The
+worker will then run the increment function and store the function's output in
+the worker's local object store.
+
+At that point, the scheduler will be notified that the outputs of the task are
+ready, and tasks that depend on those outputs can be scheduled.
+
+## Serializable Types
+
+It is not reasonable to store arbitrary Python objects in the object store or to
+ship arbitrary Python objects between machines (for example a file handle on one
+machine may not be meaningful on another). Currently, we serialize only specific
+types in the object store. **The serializable types are:**
+
+1. Primitive data types (for example, `1`, `1.0`, `"hello"`, `True`)
+2. Numpy arrays
+3. Object references
+4. Lists, tuples, and dictionaries of other serializable types, but excluding
+custom classes (for example, `[1, 1.0, "hello"]`, `{True: "hi", 1: ["hi"]}`)
+5. Custom classes where the user has provided `serialize` and `desererialize`
+methods
+
+If you wish to define a custom class and to allow it to be serialized in the
+object store, you must implement `serialize` and `deserialize` methods which
+convert the object to and from primitive data types. A simple example is shown
+below.
+
+```python
+BLOCK_SIZE = 1000
+
+class ExampleClass(object):
+  def __init__(self, field1, field2):
+    # This example assumes that field1 and field2 are serializable types.
+    self.field1 = field1
+    self.field2 = field2
+
+  def deserialize(self, primitives):
+    (field1, field2) = primitives
+    self.field1 = field1
+    self.field2 = field2
+
+  def serialize(self):
+    return (self.field1, self.field2)
+```
diff --git a/doc/tutorial.md b/doc/tutorial.md
index 4feb74f16..53029da55 100644
--- a/doc/tutorial.md
+++ b/doc/tutorial.md
@@ -1,118 +1,265 @@
-## Tutorial
+# Tutorial
 
-This section assumes that Ray has been built. See the [instructions for
-installing Ray](download-and-setup.md)
+To use Ray, you need to understand the following:
 
-### Trying it out
+- How Ray uses object references to represent immutable remote objects.
+- How Ray constructs a computation graph using remote functions.
 
-Start a shell by running this command.
-```
-python scripts/shell.py
-```
-By default, this will start up several things
+## Overview
 
-- 1 scheduler (for assigning tasks to workers)
-- 1 object store (for sharing objects between worker processes)
-- 10 workers (for executing tasks)
-- 1 driver (for submitting tasks to the scheduler)
+Ray is a distributed extension of Python. When using Ray, several processes are
+involved.
 
-Each of the above items, and each worker, is its own process.
+- A **scheduler**: The scheduler assigns tasks to workers. It is its own
+process.
+- Multiple **workers**: Workers execute tasks and store the results in object
+stores. Each worker is a separate process.
+- One **object store** per node: The object store enables the sharing of Python
+objects between worker processes so each worker does not have to have a separate
+copy.
+- A **driver**: The driver is the Python process that the user controls and
+which submits tasks to the scheduler. For example, if the user is running a
+script or using a Python shell, then the driver is the process that runs the
+script or the shell.
 
-The shell that you just started is the driver process.
+## Starting Ray
 
-You can take a Python object and store it in the object store using `ray.put`.
-This turns it into a **remote object** (we are currently on a single machine,
-but the terminology makes sense if we are on a cluster), and allows it to be
-shared among the worker processes. The function `ray.put` returns an object
-reference that is used to identify this remote object.
+To start Ray, start Python, and run the following commands.
 
 ```python
->>> xref = ray.put([1, 2, 3])
+import ray
+ray.services.start_ray_local(num_workers=10)
 ```
 
-We can use `ray.get` to retrieve the object corresponding to an object
-reference.
+That command starts a scheduler, one object store, and ten workers. Each of
+these are distinct processes. They will be killed when you exit the Python
+interpreter. They can also be killed manually with the following command.
+
+```
+killall scheduler objstore python
+```
+
+## Immutable remote objects
+
+In Ray, we can create and manipulate objects. We refer to these objects as
+**remote objects**, and we use **object references** to refer to them. Remote
+objects are stored in **object stores**, and there is one object store per node
+in the cluster. In the cluster setting, we may not actually know which machine
+each object lives on.
+
+An **object reference** is essentially a unique ID that can be used to refer to
+a remote object. If you're familiar with Futures, our object references are
+conceptually similar.
+
+We assume that remote objects are immutable. That is, their values cannot be
+changed after creation. This allows remote objects to be replicated in multiple
+object stores without needing to synchronize the copies.
+
+### Put and Get
+
+The commands `ray.get` and `ray.put` can be used to convert between Python
+objects and object references, as shown in the example below.
 
 ```python
->>> ray.get(xref)
-[1, 2, 3]
-```
-We can call a remote function.
-```python
->>> ref = example_functions.increment(1)
->>>ray.get(ref)
-2
+x = [1, 2, 3]
+ray.put(x)  # prints <ray.ObjRef at 0x1031baef0>
 ```
 
-Note that `example_functions.increment` is defined in
-[`scripts/example_functions.py`](../scripts/example_functions.py) as
+The command `ray.put(x)` would be run by a worker process or by the driver
+process (the driver process is the one running your script). It takes a Python
+object and copies it to the local object store (here *local* means *on the same
+node*). Once the object has been stored in the object store, its value cannot be
+changed.
+
+In addition, `ray.put(x)` returns an object reference, which is essentially an
+ID that can be used to refer to the newly created remote object. If we save the
+object reference in a variable with `ref = ray.put(x)`, then we can pass `ref`
+into remote functions, and those remote functions will operate on the
+corresponding remote object.
+
+The command `ray.get(ref)` takes an object reference and creates a Python object
+from the corresponding remote object. For some objects like arrays, we can use
+shared memory and avoid copying the object. For other objects, this currently
+copies the object from the object store into the memory of the worker process.
+If the remote object corresponding to the object reference `ref` does not live
+on the same node as the worker that calls `ray.get(ref)`, then the remote object
+will first be copied from an object store that has it to the object store that
+needs it.
+
+```python
+>>> ref = ray.put([1, 2, 3])
+>>> ray.get(ref)  # prints [1, 2, 3]
+```
+
+If the remote object corresponding to the object reference `ref` has not been
+created yet, *the command `ray.get(ref)` will wait until the remote object has
+been created.*
+
+## Computation graphs in Ray
+
+Ray represents computation with a directed acyclic graph of tasks. Tasks are
+added to this graph by calling **remote functions**.
+
+For example, a normal Python function looks like this.
+```python
+def add(a, b):
+  return a + b
+```
+A remote function in Ray looks like this.
+```python
+@ray.remote([int, int], [int])
+def add(a, b):
+  return a + b
+```
+
+The information passed to the `@ray.remote` decorator includes type information
+for the arguments and for the return values of the function. Because of the
+distinction that we make between *submitting a task* and *executing the task*,
+we require type information so that we can catch type errors when the remote
+function is called instead of catching them when the task is actually executed
+(which could be much later and could be on a different machine).
+
+### Remote functions
+
+Whereas in regular Python, calling `add(1, 2)` would return `3`, in Ray, calling
+`add(1, 2)` does not actually execute the task. Instead, it adds a task to the
+computation graph and immediately returns an object reference to the output of
+the computation.
+
+```python
+>>> ref = add(1, 2)
+>>> ray.get(ref)  # prints 3
+```
+
+There is a sharp distinction between *submitting a task* and *executing the
+task*. When a remote function is called, the task of executing that function is
+submitted to the scheduler, and the scheduler immediately returns object
+references for the outputs of the task. However, the task will not be executed
+until the scheduler actually schedules the task on a worker.
+
+When a task is submitted, each argument may be passed in by value or by object
+reference. For example, these lines have the same behavior.
+
+```python
+>>> add(1, 2)
+>>> add(1, ray.put(2))
+>>> add(ray.put(1), ray.put(2))
+```
+
+Remote functions never return actual values, they always return object
+references.
+
+When the remote function is actually executed, it operates on Python objects.
+That is, if the remote function was called with any object references, the
+Python objects corresponding to those object references will be retrieved and
+passed into the actual execution of the remote function.
+
+### Blocking computation
+
+In a regular Python script, the specification of a computation is intimately
+linked to the actual execution of the code. For example, consider the following
+code.
+
+```python
+import time
+
+# This takes 50 seconds.
+for i in range(10):
+  time.sleep(5)
+```
+
+At the core of the above script, there are ten separate tasks, each of which
+sleeps for five seconds (this is a toy example, but you could imagine replacing
+the call to `sleep` with some computationally intensive task). These tasks do
+not depend on each other, so in principle, they could be executed in parallel.
+However, in the above implementation, they will be executed serially, which will
+take fifty seconds.
+
+Ray gets around this by representing computation as a graph of tasks, where some
+tasks depend on the outputs of other tasks and where tasks can be executed once
+their dependencies are done.
+
+For example, suppose we define the remote function `sleep` to be a wrapper
+around `time.sleep`.
 
 ```python
 @ray.remote([int], [int])
-def increment(x):
-  return x + 1
+def sleep(n):
+  time.sleep(n)
+  return 0
 ```
 
-Note that, we can pass arguments into remote functions either by value or by
-object reference. That is, these two lines have the same behavior.
+Then we can write
 
 ```python
->>> ray.get(example_functions.increment(1))
-2
->>> ray.get(example_functions.increment(ray.put(1)))
-2
-```
-This is convenient for chaining remote functions together, for example,
-```python
->>> ref = example_functions.increment(1)
->>> ref = example_functions.increment(ref)
->>> ref = example_functions.increment(ref)
->>> ray.get(ref)
-4
+# Submit ten tasks to the scheduler. This finishes almost immediately.
+result_refs = []
+for i in range(10):
+  result_refs.append(sleep(5))
+
+# Wait for the results. If we have at least ten workers, this takes 5 seconds.
+[ray.get(ref) for ref in result_refs]  # prints [0, 0, 0, 0, 0, 0, 0, 0, 0, 0]
 ```
 
-### Visualize the computation graph
+The for loop simply adds ten tasks to the computation graph, with no
+dependencies between the tasks. It executes almost instantaneously. Afterwards,
+we use `ray.get` to wait for the tasks to finish. If we have at least ten
+workers, then all ten tasks can be executed in parallel, and so the overall
+script should take five seconds.
+
+### Visualizing the Computation Graph
+
+The computation graph can be viewed as follows.
 
-At any point, we can visualize the computation graph by running
 ```python
->>> ray.visualize_computation_graph(view=True)
+ray.visualize_computation_graph(view=True)
 ```
-This will display an image like the following one.
 
 <p align="center">
-  <img src="figures/compgraph1.png" width="300">
+  <img src="figures/compgraph1.png">
 </p>
 
-### Restart workers
+In this figure, boxes are tasks and ovals are objects.
 
-During development, suppose that you want to change the implementation of
-`example_functions.increment`, but you've already done a bunch of work in the
-shell loading and preprocessing data, and you don't want to have to recompute
-all of that work.
+The box that says `op-root` in it just refers to the overall script itself. The
+dotted lines indicate that the script launched 10 tasks (tasks are denoted by
+rectangular boxes). The solid lines indicate that each task produces one output
+(represented by an oval).
 
-We can simply restart the workers.
+It is clear from the computation graph that these ten tasks can be executed in
+parallel.
 
-First, change the code, for example, modify the function
-`example_functions.increment` in
-[`scripts/example_functions.py`](../scripts/example_functions.py) to add 10
-instead of 1.
+Computation graphs encode dependencies. For example, suppose we define
 
 ```python
-@ray.remote([int], [int])
-def increment(x):
-  return x + 10
-```
-Then from the shell, restart the workers like this.
-```python
->>> ray.restart_workers("scripts/example_worker.py") # This should be the correct relative path to the example_worker.py code
-```
-We can check that the code has been updated by running.
-```python
->>> ray.get(example_functions.increment(1))
-11
+import numpy as np
+
+@ray.remote([list], [np.ndarray])
+def zeros(shape):
+  return np.zeros(shape)
+
+@ray.remote([np.ndarray, np.ndarray], [np.ndarray])
+def dot(a, b):
+  return np.dot(a, b)
 ```
 
-Note that it is not as simple as running `reload(example_functions)` because we
-need to reload the Python module on all of the workers as well, and the workers
-are separate Python processes. Calling `reload(example_functions)` would only
-reload the module on the driver.
+Then we run
+
+```python
+aref = zeros([10, 10])
+bref = zeros([10, 10])
+cref = dot(aref, bref)
+```
+
+The corresponding computation graph looks like this.
+
+<p align="center">
+  <img src="figures/compgraph2.png" width="300">
+</p>
+
+The three dashed lines indicate that the script launched three tasks (the two
+`zeros` tasks and the one `dot` task). Each task produces a single output, and
+the `dot` task depends on the outputs of the two `zeros` tasks.
+
+This makes it clear that the two `zeros` tasks can execute in parallel but that
+the `dot` task must wait until the two `zeros` tasks have finished.
diff --git a/doc/using-ray-on-a-cluster.md b/doc/using-ray-on-a-cluster.md
index 3edb9dd85..cc2b970c3 100644
--- a/doc/using-ray-on-a-cluster.md
+++ b/doc/using-ray-on-a-cluster.md
@@ -1,4 +1,4 @@
-## Using Ray on a cluster
+# Using Ray on a cluster
 
 Running Ray on a cluster is still experimental.
 
@@ -6,12 +6,12 @@ Ray can be used in several ways. In addition to running on a single machine, Ray
 is designed to run on a cluster of machines. This document is about how to use
 Ray on a cluster.
 
-### Launching a cluster on EC2
+## Launching a cluster on EC2
 
 This section describes how to start a cluster on EC2. These instructions are
 copied and adapted from https://github.com/amplab/spark-ec2.
 
-#### Before you start
+### Before you start
 
 - Create an Amazon EC2 key pair for yourself. This can be done by logging into
 your Amazon Web Services account through the [AWS
@@ -25,7 +25,7 @@ and secret access key. These can be obtained from the [AWS
 homepage](http://aws.amazon.com/) by clicking Account > Security Credentials >
 Access Credentials.
 
-#### Launching a Cluster
+### Launching a Cluster
 
 - Go into the `ray/scripts` directory.
 - Run `python ec2.py -k <keypair> -i <key-file> -s <num-slaves> launch
@@ -56,7 +56,7 @@ capacity in one zone, and you should try to launch in another.
 Instances](http://aws.amazon.com/ec2/spot-instances/), bidding for the given
 maximum price (in dollars).
 
-### Getting started with Ray on a cluster
+## Getting started with Ray on a cluster
 
 These instructions work on EC2, but they may require some modifications to run
 on your own cluster. In particular, on EC2, running `sudo` does not require a
@@ -125,8 +125,6 @@ For example,
     - `cluster.update_ray()` - This pulls the latest Ray source code and builds
       it.
 
-### Running Ray on a cluster
-
 Once you've started a Ray cluster using the above instructions, to actually use
 Ray, ssh to the head node (the first node listed in the `nodes.txt` file) and
 attach a shell to the already running cluster.
diff --git a/examples/hyperopt/README.md b/examples/hyperopt/README.md
index 578b9a5ae..ed7f68806 100644
--- a/examples/hyperopt/README.md
+++ b/examples/hyperopt/README.md
@@ -1,4 +1,4 @@
-## Hyperparameter Optimization
+# Hyperparameter Optimization
 
 This document provides a walkthrough of the hyperparameter optimization example.
 To run the application, first install this dependency.
@@ -22,7 +22,7 @@ Choosing these parameters can be challenging, and so a common practice is to
 search over the space of hyperparameters. One approach that works surprisingly
 well is to randomly sample different options.
 
-### The serial version
+## The serial version
 
 Suppose that we want to train a convolutional network, but we aren't sure how to
 choose the following hyperparameters:
@@ -74,7 +74,7 @@ Of course, as there are no dependencies between the different invocations of
 `train_cnn_and_compute_accuracy`, this computation could easily be parallelized
 over multiple cores or multiple machines. Let's do that now.
 
-### The distributed version
+## The distributed version
 
 First, let's turn `train_cnn_and_compute_accuracy` into a remote function in Ray
 by writing it as follows. In this example application, a slightly more
@@ -115,7 +115,7 @@ their values with `ray.get`.
 results = [(params, ray.get(ref)) for (params, ref) in result_refs]
 ```
 
-### Additional notes
+## Additional notes
 
 **Early Stopping:** Sometimes when running an optimization, it is clear early on
 that the hyperparameters being used are bad (for example, the loss function may
diff --git a/examples/lbfgs/README.md b/examples/lbfgs/README.md
index 2b48bdea1..2d4400e85 100644
--- a/examples/lbfgs/README.md
+++ b/examples/lbfgs/README.md
@@ -1,4 +1,4 @@
-## Batch L-BFGS
+# Batch L-BFGS
 
 This document provides a walkthrough of the L-BFGS example. To run the
 application, first install these dependencies.
@@ -21,7 +21,7 @@ one such algorithm. It is a quasi-Newton method that uses gradient information
 to approximate the inverse Hessian of the loss function in a computationally
 efficient manner.
 
-### The serial version
+## The serial version
 
 First we load the data in batches. Here, each element in `batches` is a tuple
 whose first component is a batch of `100` images and whose second component is a
@@ -73,7 +73,7 @@ theta_init = 1e-2 * np.random.normal(size=dim)
 result = scipy.optimize.fmin_l_bfgs_b(full_loss, theta_init, fprime=full_grad)
 ```
 
-### The distributed version
+## The distributed version
 
 In this example, the computation of the gradient itself can be done in parallel
 on a number of workers or machines.