From 054a046b691a74d838dfed9228614a13d6019948 Mon Sep 17 00:00:00 2001 From: Robert Nishihara Date: Fri, 24 Mar 2017 17:33:26 -0700 Subject: [PATCH] Fix installation instructions on Ubuntu and convert md -> rst. (#389) --- doc/source/index.rst | 8 +- doc/source/install-on-docker.md | 151 ---------------- doc/source/install-on-docker.rst | 188 ++++++++++++++++++++ doc/source/install-on-macosx.md | 64 ------- doc/source/install-on-macosx.rst | 76 ++++++++ doc/source/install-on-ubuntu.md | 65 ------- doc/source/install-on-ubuntu.rst | 81 +++++++++ doc/source/installation-troubleshooting.md | 31 ---- doc/source/installation-troubleshooting.rst | 42 +++++ 9 files changed, 391 insertions(+), 315 deletions(-) delete mode 100644 doc/source/install-on-docker.md create mode 100644 doc/source/install-on-docker.rst delete mode 100644 doc/source/install-on-macosx.md create mode 100644 doc/source/install-on-macosx.rst delete mode 100644 doc/source/install-on-ubuntu.md create mode 100644 doc/source/install-on-ubuntu.rst delete mode 100644 doc/source/installation-troubleshooting.md create mode 100644 doc/source/installation-troubleshooting.rst diff --git a/doc/source/index.rst b/doc/source/index.rst index fe26fd598..c9e74a558 100644 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -7,10 +7,10 @@ Ray :maxdepth: 1 :caption: Installation - install-on-ubuntu.md - install-on-macosx.md - install-on-docker.md - installation-troubleshooting.md + install-on-ubuntu.rst + install-on-macosx.rst + install-on-docker.rst + installation-troubleshooting.rst .. toctree:: :maxdepth: 1 diff --git a/doc/source/install-on-docker.md b/doc/source/install-on-docker.md deleted file mode 100644 index 3b96ed382..000000000 --- a/doc/source/install-on-docker.md +++ /dev/null @@ -1,151 +0,0 @@ -# Installation on Docker - -You can install Ray on any platform that runs Docker. We do not presently publish Docker images for Ray, but you can build them yourself using the Ray distribution. - -Using Docker can streamline the build process and provide a reliable way to get up and running quickly. - -## Install Docker - -### Mac, Linux, Windows platforms - -The Docker Platform release is available for Mac, Windows, and Linux platforms. Please download the appropriate version from the [Docker website](https://www.docker.com/products/overview#/install_the_platform) and follow the corresponding installation instructions. -Linux user may find these [alternate instructions](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04) helpful. - -### Docker installation on EC2 with Ubuntu - -The instructions below show in detail how to prepare an Amazon EC2 instance running Ubuntu 16.04 for use with Docker. - -Apply initialize the package repository and apply system updates: - -``` -sudo apt-get update -sudo apt-get -y dist-upgrade -``` - -Install Docker and start the service: -``` -sudo apt-get install -y docker.io -sudo service docker start -``` - -Add the `ubuntu` user to the `docker` group to allow running Docker commands without sudo: -``` -sudo usermod -a -G docker ubuntu -``` - -Initiate a new login to gain group permissions (alternatively, log out and log back in again): - -``` -exec sudo su -l ubuntu -``` - -Confirm that docker is running: - -``` -docker images -``` -Should produce an empty table similar to the following: -``` -REPOSITORY TAG IMAGE ID CREATED SIZE -``` - - -## Clone the Ray repository - -``` -git clone https://github.com/ray-project/ray.git -``` - -## Build Docker images - -Run the script to create Docker images. - -``` -cd ray -./build-docker.sh -``` - -This script creates several Docker images: - - * The `ray-project/deploy` image is a self-contained copy of code and binaries suitable for end users. - * The `ray-project/examples` adds additional libraries for running examples. - * The `ray-project/base-deps` image builds from Ubuntu Xenial and includes Anaconda and other basic dependencies and can serve as a starting point for developers. - -Review images by listing them: -``` -$ docker images -``` - -Output should look something like the following: -``` -REPOSITORY TAG IMAGE ID CREATED SIZE -ray-project/examples latest 7584bde65894 4 days ago 3.257 GB -ray-project/deploy latest 970966166c71 4 days ago 2.899 GB -ray-project/base-deps latest f45d66963151 4 days ago 2.649 GB -ubuntu xenial f49eec89601e 3 weeks ago 129.5 MB -``` - - -## Launch Ray in Docker - -Start out by launching the deployment container. - -``` -docker run --shm-size= -t -i ray-project/deploy -``` - -Replace `` with a limit appropriate for your system, for example `512M` or `2G`. -The `-t` and `-i` options here are required to support interactive use of the container. - -**Note:** Ray requires a **large** amount of shared memory because each object -store keeps all of its objects in shared memory, so the amount of shared memory -will limit the size of the object store. - -You should now see a prompt that looks something like: - -``` -root@ebc78f68d100:/ray# -``` - - -## Test if the installation succeeded - -To test if the installation was successful, try running some tests. Within the container shell enter the following commands: - -``` -python test/runtest.py # This tests basic functionality. -python test/array_test.py # This tests some array libraries. -``` - -You are now ready to continue with the [Tutorial](tutorial.md). - -## Running examples in Docker - -Ray includes a Docker image that includes dependencies necessary for running some of the examples. This can be an easy way to see Ray in action on a variety of workloads. - -Launch the examples container. -``` -docker run --shm-size=1024m -t -i ray-project/examples -``` - -### Hyperparameter optimization - - -``` -cd /ray/examples/hyperopt/ -python driver.py -``` - -### Batch L-BFGS - -``` -cd /ray/examples/lbfgs/ -python driver.py -``` - -### Learning to play Pong - -``` -cd /ray/examples/rl_pong/ -python driver.py -``` diff --git a/doc/source/install-on-docker.rst b/doc/source/install-on-docker.rst new file mode 100644 index 000000000..9fa245c16 --- /dev/null +++ b/doc/source/install-on-docker.rst @@ -0,0 +1,188 @@ +Installation on Docker +====================== + +You can install Ray on any platform that runs Docker. We do not presently +publish Docker images for Ray, but you can build them yourself using the Ray +distribution. + +Using Docker can streamline the build process and provide a reliable way to get +up and running quickly. + +Install Docker +-------------- + +Mac, Linux, Windows platforms +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The Docker Platform release is available for Mac, Windows, and Linux platforms. +Please download the appropriate version from the `Docker website`_ and follow +the corresponding installation instructions. Linux user may find these +`alternate instructions`_ helpful. + +.. _`Docker website`: https://www.docker.com/products/overview#/install_the_platform +.. _`alternate instructions`: https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04 + +Docker installation on EC2 with Ubuntu +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The instructions below show in detail how to prepare an Amazon EC2 instance +running Ubuntu 16.04 for use with Docker. + +Apply initialize the package repository and apply system updates: + +.. code-block:: bash + + sudo apt-get update + sudo apt-get -y dist-upgrade + +Install Docker and start the service: + +.. code-block:: bash + + sudo apt-get install -y docker.io + sudo service docker start + + +Add the ``ubuntu`` user to the ``docker`` group to allow running Docker commands +without sudo: + +.. code-block:: bash + + sudo usermod -a -G docker ubuntu + +Initiate a new login to gain group permissions (alternatively, log out and log +back in again): + +.. code-block:: bash + + exec sudo su -l ubuntu + +Confirm that docker is running: + +.. code-block:: bash + + docker images + +Should produce an empty table similar to the following: + +.. code-block:: bash + + REPOSITORY TAG IMAGE ID CREATED SIZE + + +Clone the Ray repository +------------------------ + +.. code-block:: bash + + git clone https://github.com/ray-project/ray.git + + +Build Docker images +------------------- + +Run the script to create Docker images. + +.. code-block:: bash + + cd ray + ./build-docker.sh + +This script creates several Docker images: + +- The ``ray-project/deploy`` image is a self-contained copy of code and binaries + suitable for end users. +- The ``ray-project/examples`` adds additional libraries for running examples. +- The ``ray-project/base-deps`` image builds from Ubuntu Xenial and includes + Anaconda and other basic dependencies and can serve as a starting point for + developers. + +Review images by listing them: + +.. code-block:: bash + + docker images + +Output should look something like the following: + +.. code-block:: bash + + REPOSITORY TAG IMAGE ID CREATED SIZE + ray-project/examples latest 7584bde65894 4 days ago 3.257 GB + ray-project/deploy latest 970966166c71 4 days ago 2.899 GB + ray-project/base-deps latest f45d66963151 4 days ago 2.649 GB + ubuntu xenial f49eec89601e 3 weeks ago 129.5 MB + + +Launch Ray in Docker +-------------------- + +Start out by launching the deployment container. + +.. code-block:: bash + + docker run --shm-size= -t -i ray-project/deploy + +Replace ```` with a limit appropriate for your system, for example +``512M`` or ``2G``. The ``-t`` and ``-i`` options here are required to support +interactive use of the container. + +**Note:** Ray requires a **large** amount of shared memory because each object +store keeps all of its objects in shared memory, so the amount of shared memory +will limit the size of the object store. + +You should now see a prompt that looks something like: + +.. code-block:: bash + + root@ebc78f68d100:/ray# + + +Test if the installation succeeded +---------------------------------- + +To test if the installation was successful, try running some tests. Within the +container shell enter the following commands: + +.. code-block:: bash + + python test/runtest.py # This tests basic functionality. + +You are now ready to continue with the `tutorial`_. + +.. _`tutorial`: http://ray.readthedocs.io/en/latest/tutorial.html + +Running examples in Docker +-------------------------- + +Ray includes a Docker image that includes dependencies necessary for running +some of the examples. This can be an easy way to see Ray in action on a variety +of workloads. + +Launch the examples container. + +.. code-block:: bash + + docker run --shm-size=1024m -t -i ray-project/examples + +Hyperparameter optimization +~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + cd /ray/examples/hyperopt/ + python /ray/examples/hyperopt/hyperopt_simple.py + +Batch L-BFGS +~~~~~~~~~~~~ + +.. code-block:: bash + + python /ray/examples/lbfgs/driver.py + +Learning to play Pong +~~~~~~~~~~~~~~~~~~~~~ + +.. code-block:: bash + + python /ray/examples/rl_pong/driver.py diff --git a/doc/source/install-on-macosx.md b/doc/source/install-on-macosx.md deleted file mode 100644 index 8cf85f653..000000000 --- a/doc/source/install-on-macosx.md +++ /dev/null @@ -1,64 +0,0 @@ -# Installation on Mac OS X - -Ray should work with Python 2 and Python 3. We have tested Ray on OS X 10.11. - -## Dependencies - -To install Ray, first install the following dependencies. We recommend using -[Anaconda](https://www.continuum.io/downloads). - -``` -brew update -brew install cmake automake autoconf libtool boost wget - -pip install numpy cloudpickle funcsigs colorama psutil redis flatbuffers --ignore-installed six -``` - -If you are using Anaconda, you may also need to run the following. - -``` -conda install libgcc -``` - -## Install Ray - -Ray can be built from the repository as follows. - -``` -git clone https://github.com/ray-project/ray.git -cd ray/python -python setup.py install --user -``` - -## Test if the installation succeeded - -To test if the installation was successful, try running some tests. This assumes -that you've cloned the git repository. - -``` -python test/runtest.py -``` - -## Optional - web UI - -Ray's web UI requires **Python 3**. To enable the web UI to work, install these -Python packages. - -``` -pip install aioredis asyncio websockets -``` - -Then install -[polymer](https://www.polymer-project.org/1.0/docs/tools/polymer-cli), which -also requires [Node.js](https://nodejs.org/en/download/) and -[Bower](http://bower.io/#install-bower). - -Once you've installed Polymer, run the following. - -``` -cd ray/webui -bower install -``` - -Then while Ray is running, you should be able to view the web UI at -`http://localhost:8080`. diff --git a/doc/source/install-on-macosx.rst b/doc/source/install-on-macosx.rst new file mode 100644 index 000000000..712521680 --- /dev/null +++ b/doc/source/install-on-macosx.rst @@ -0,0 +1,76 @@ +Installation on Mac OS X +======================== + +Ray should work with Python 2 and Python 3. We have tested Ray on OS X 10.11 and +10.12. + +Dependencies +------------ + +To install Ray, first install the following dependencies. We recommend using +`Anaconda`_. + +.. _`Anaconda`: https://www.continuum.io/downloads + +.. code-block:: bash + + brew update + brew install cmake automake autoconf libtool boost wget + + pip install numpy cloudpickle funcsigs colorama psutil redis flatbuffers --ignore-installed six + +If you are using Anaconda, you may also need to run the following. + +.. code-block:: bash + + conda install libgcc + + +Install Ray +----------- + +Ray can be built from the repository as follows. + +.. code-block:: bash + + git clone https://github.com/ray-project/ray.git + cd ray/python + python setup.py install --user + + +Test if the installation succeeded +---------------------------------- + +To test if the installation was successful, try running some tests. This assumes +that you've cloned the git repository. + +.. code-block:: bash + + python test/runtest.py + + +Optional - web UI +----------------- + +Ray's web UI requires **Python 3**. To enable the web UI to work, install these +Python packages. + +.. code-block:: bash + + pip install aioredis asyncio websockets + +Then install `Polymer`_, which also requires `Node.js`_ and `Bower`_. + +.. _`Polymer`: https://www.polymer-project.org/1.0/docs/tools/polymer-cli +.. _`Node.js`: https://nodejs.org/en/download/ +.. _`Bower`: http://bower.io/#install-bower + +Once you've installed Polymer, run the following. + +.. code-block:: bash + + cd ray/webui + bower install + +Then while Ray is running, you should be able to view the web UI at +``http://localhost:8080``. diff --git a/doc/source/install-on-ubuntu.md b/doc/source/install-on-ubuntu.md deleted file mode 100644 index 31ac80296..000000000 --- a/doc/source/install-on-ubuntu.md +++ /dev/null @@ -1,65 +0,0 @@ -# Installation on Ubuntu - -Ray should work with Python 2 and Python 3. We have tested Ray on Ubuntu 14.04 -and Ubuntu 16.04 - -## Dependencies - -To install Ray, first install the following dependencies. We recommend using -[Anaconda](https://www.continuum.io/downloads). - -``` -sudo apt-get update -sudo apt-get install -y cmake build-essential autoconf curl libtool libboost-all-dev unzip python-dev python-pip # If you're using Anaconda, then python-dev and python-pip are unnecessary. - -pip install numpy cloudpickle funcsigs colorama psutil redis flatbuffers -``` - -If you are using Anaconda, you may also need to run the following. - -``` -conda install libgcc -``` - -## Install Ray - -Ray can be built from the repository as follows. - -``` -git clone https://github.com/ray-project/ray.git -cd ray/python -python setup.py install --user -``` - -## Test if the installation succeeded - -To test if the installation was successful, try running some tests. This assumes -that you've cloned the git repository. - -``` -python test/runtest.py -``` - -## Optional - web UI - -Ray's web UI requires **Python 3**. To enable the web UI to work, install these -Python packages. - -``` -pip install aioredis asyncio websockets -``` - -Then install -[polymer](https://www.polymer-project.org/1.0/docs/tools/polymer-cli), which -also requires [Node.js](https://nodejs.org/en/download/) and -[Bower](http://bower.io/#install-bower). - -Once you've installed Polymer, run the following. - -``` -cd ray/webui -bower install -``` - -Then while Ray is running, you should be able to view the web UI at -`http://localhost:8080`. diff --git a/doc/source/install-on-ubuntu.rst b/doc/source/install-on-ubuntu.rst new file mode 100644 index 000000000..ae90ad241 --- /dev/null +++ b/doc/source/install-on-ubuntu.rst @@ -0,0 +1,81 @@ +Installation on Ubuntu +====================== + +Ray should work with Python 2 and Python 3. We have tested Ray on Ubuntu 14.04 +and Ubuntu 16.04. + +Dependencies +------------ + +To install Ray, first install the following dependencies. We recommend using +`Anaconda`_. + +.. _`Anaconda`: https://www.continuum.io/downloads + +.. code-block:: bash + + sudo apt-get update + sudo apt-get install -y cmake build-essential autoconf curl libtool libboost-all-dev unzip + + # If you are not using Anaconda, you need the following. + sudo apt-get install python-dev # For Python 2. + sudo apt-get install python3-dev # For Python 3. + + pip install numpy cloudpickle funcsigs colorama psutil redis flatbuffers + + +If you are using Anaconda, you may also need to run the following. + +.. code-block:: bash + + conda install libgcc + + +Install Ray +----------- + +Ray can be built from the repository as follows. + +.. code-block:: bash + + git clone https://github.com/ray-project/ray.git + cd ray/python + python setup.py install --user + + +Test if the installation succeeded +---------------------------------- + +To test if the installation was successful, try running some tests. This assumes +that you've cloned the git repository. + +.. code-block:: bash + + python test/runtest.py + + +Optional - web UI +----------------- + +Ray's web UI requires **Python 3**. To enable the web UI to work, install these +Python packages. + +.. code-block:: bash + + pip install aioredis asyncio websockets + +Then install `Polymer`_, which also requires `Node.js`_ and `Bower`_. + +.. _`Polymer`: https://www.polymer-project.org/1.0/docs/tools/polymer-cli +.. _`Node.js`: https://nodejs.org/en/download/ +.. _`Bower`: http://bower.io/#install-bower + +Once you've installed Polymer, run the following. + +.. code-block:: bash + + cd ray/webui + bower install + +Then while Ray is running, you should be able to view the web UI at +``http://localhost:8080``. diff --git a/doc/source/installation-troubleshooting.md b/doc/source/installation-troubleshooting.md deleted file mode 100644 index 70045db2e..000000000 --- a/doc/source/installation-troubleshooting.md +++ /dev/null @@ -1,31 +0,0 @@ -# Installation Troubleshooting - -## Trouble installing Numbuf - -### Arrow fails to build -If the installation of Numbuf fails, chances are there was a problem building -Arrow. Some candidate possibilities. - -#### You have a different version of Flatbuffers installed - -Arrow pulls and builds its own copy of Flatbuffers, but if you already have -Flatbuffers installed, Arrow may find the wrong version. If a directory like -`/usr/local/include/flatbuffers` shows up in the output when installing Numbuf, -this may be the problem. To solve it, get rid of the old version of flatbuffers. - -#### There is some problem with Boost - -If a message like `Unable to find the requested Boost libraries` appears when -installing Numbuf, there may be a problem with Boost. This can happen if you -installed Boost using MacPorts. This is sometimes solved by using Brew instead. - -## Trouble installing or running Ray - -### One of the Ray libraries is compiled against the wrong version of Python - -If there is a segfault or a sigabort immediately upon importing Ray, one of the -components may have been compiled against the wrong Python libraries. CMake -should normally find the right version of Python, but this process is not -completely reliable. In this case, check the CMake output from installation and -make sure that the version of the Python libraries that were found match the -version of Python that you're using. diff --git a/doc/source/installation-troubleshooting.rst b/doc/source/installation-troubleshooting.rst new file mode 100644 index 000000000..e0a033250 --- /dev/null +++ b/doc/source/installation-troubleshooting.rst @@ -0,0 +1,42 @@ +Installation Troubleshooting +============================ + +Trouble installing Numbuf +------------------------- + +If the installation of Numbuf fails, chances are there was a problem building +Arrow. Some candidate possibilities. + +You have a different version of Flatbuffers installed +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +Arrow pulls and builds its own copy of Flatbuffers, but if you already have +Flatbuffers installed, Arrow may find the wrong version. If a directory like +``/usr/local/include/flatbuffers`` shows up in the output when installing +Numbuf, this may be the problem. To solve it, get rid of the old version of +flatbuffers. + +There is some problem with Boost +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If a message like ``Unable to find the requested Boost libraries`` appears when +installing Numbuf, there may be a problem with Boost. This can happen if you +installed Boost using MacPorts. This is sometimes solved by using Brew instead. + +Trouble installing or running Ray +--------------------------------- + +One of the Ray libraries is compiled against the wrong version of Python +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +If there is a segfault or a sigabort immediately upon importing Ray, one of the +components may have been compiled against the wrong Python libraries. CMake +should normally find the right version of Python, but this process is not +completely reliable. In this case, check the CMake output from installation and +make sure that the version of the Python libraries that were found match the +version of Python that you're using. + +Note that it's common to have multiple versions of Python on your machine (for +example both Python 2 and Python 3). Ray will be compiled against whichever +version of Python is found when you run the ``python`` command from the +command line, so make sure this is the version you wish to use.