[Projects] Add small tutorial for projects (#6641)

This commit is contained in:
Philipp Moritz
2020-01-20 09:33:41 -08:00
committed by GitHub
parent 10609c3a19
commit 96e2c1ae74
3 changed files with 135 additions and 1 deletions
+96
View File
@@ -38,6 +38,102 @@ for instructions on how to run these examples:
- `PyTorch Transformers <https://github.com/ray-project/ray/blob/master/python/ray/projects/examples/pytorch-transformers/ray-project/project.yaml>`__:
A library of state-of-the-art pretrained models for Natural Language Processing (NLP)
Tutorial
--------
We will walk through how to use projects by executing the `streaming MapReduce example <auto_examples/plot_streaming.html>`_.
Commands always apply to the project in the current directory.
Let us switch into the project directory with
.. code-block:: bash
cd ray/doc/examples/streaming
A session represents a running instance of a project. Let's start one with
.. code-block:: bash
ray session start
The ``ray session start`` command
will bring up a new cluster and initialize the environment of the cluster
according to the `environment` section of the `project.yaml`, installing all
dependencies of the project.
Now we can execute a command in the session. To see a list of all available
commands of the project, run
.. code-block:: bash
ray session commands
which produces the following output:
.. code-block::
Active project: ray-example-streaming
Command "run":
usage: run [--num-mappers NUM_MAPPERS] [--num-reducers NUM_REDUCERS]
Start the streaming example.
optional arguments:
--num-mappers NUM_MAPPERS
Number of mapper actors used
--num-reducers NUM_REDUCERS
Number of reducer actors used
As you see, in this project there is only a single ``run`` command which has arguments
``--num-mappers`` and ``--num-reducers``. We can execute the streaming
wordcount with the default parameters by running
.. code-block:: bash
ray session execute run
You can interrupt the command with ``<Control>-c`` and attach to the running session by executing
.. code-block:: bash
ray session attach --tmux
Inside the session you can for example edit the streaming applications with
.. code-block:: bash
cd ray-example-streaming
emacs streaming.py
Try for example to add the following lines after the ``for count in counts:`` loop:
.. code-block:: python
if "million" in wordcounts:
print("Found the word!")
and re-run the application from outside the session with
.. code-block:: bash
ray session execute run
The session can be terminated from outside the session with
.. code-block:: bash
ray session stop
Project file format (project.yaml)
----------------------------------