wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-18 12:20:14 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge branch 'master' into py39

Browse Source
This commit is contained in:
Akash Patel
2020-12-16 11:04:27 -05:00
committed by GitHub
parent bd866d926d 91878d18b5
commit 7d8a008aeb
489 changed files with 36844 additions and 8744 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/cluster/autoscaling.rst
+2 -2
View File
@@ -10,7 +10,7 @@ Basics
The Ray Cluster Launcher will automatically enable a load-based autoscaler. The scheduler will look at the task, actor, and placement group resource demands from the cluster, and tries to add the minimum set of nodes that can fulfill these demands. When nodes are idle for more than a timeout, they will be removed, down to the ``min_workers`` limit. The head node is never removed.
To avoid launching too many nodes at once, the number of nodes allowed to be pending is limited by the ``upscaling_speed`` setting. By default it is set to ``1.0``, which means the cluster can grow in size by at most ``100%`` at a time (doubling in size each time). This fraction can be set to as high as needed, e.g., ``99999`` to allow the cluster to quickly grow to its max size.
To avoid launching too many nodes at once, the number of nodes allowed to be pending is limited by the ``upscaling_speed`` setting. By default it is set to ``1.0``, which means the cluster can be growing in size by at most ``100%`` at any time (e.g., if the cluster currently has 20 nodes, at most 20 pending launches are allowed). This fraction can be set to as high as needed, e.g., ``99999`` to allow the cluster to quickly grow to its max size.
In more detail, the autoscaler implements the following control loop:
@@ -124,7 +124,7 @@ The node config tells the underlying Cloud provider how to launch a node of this
node_config:
InstanceType: p2.xlarge
The resources field tells the autoscaler what kinds of resources this node provides. This can include custom resources as well (e.g., "Custom2"). This field enables the autoscaler to automatically select the right kind of nodes to launch given the resource demands of the application. The resources specified here will be automatically passed to the ``ray start`` command for the node via an environment variable. For more information, see also the `resource demand scheduler <https://github.com/ray-project/ray/blob/master/python/ray/autoscaler/resource_demand_scheduler.py>`__:
The resources field tells the autoscaler what kinds of resources this node provides. This can include custom resources as well (e.g., "Custom2"). This field enables the autoscaler to automatically select the right kind of nodes to launch given the resource demands of the application. The resources specified here will be automatically passed to the ``ray start`` command for the node via an environment variable. For more information, see also the `resource demand scheduler <https://github.com/ray-project/ray/blob/master/python/ray/autoscaler/_private/resource_demand_scheduler.py>`__:
.. code-block:: yaml
doc/source/cluster/index.rst
+20 -14
View File
@@ -21,26 +21,34 @@ Clusters are started with the :ref:`Ray Cluster Launcher <ref-automatic-cluster>
You can also create a Ray cluster using a standard cluster manager such as :ref:`Kubernetes <ray-k8s-deploy>`, :ref:`YARN <ray-yarn-deploy>`, or :ref:`SLURM <ray-slurm-deploy>`.
After a cluster is started, you need to connect your program to the Ray cluster.
After a cluster is started, you need to connect your program to the Ray cluster by starting a driver process on the same node as where you ran ``ray start``:
.. tabs::
.. group-tab:: python
.. code-tab:: python
You can connect to this Ray runtime by starting a Python process that calls the following on the same node as where you ran ``ray start``:
.. code-block:: python
# This must
import ray
ray.init(address='auto')
# This must
import ray
ray.init(address='auto')
.. group-tab:: java
If you want to run Java code, you need to specify the classpath via the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
.. code-block:: java
import io.ray.api.Ray;
public class MyRayApp {
public static void main(String[] args) {
Ray.init();
...
}
}
.. code-block:: bash
$ ray start ... --code-search-path=/path/to/jars
java -classpath <classpath> \
-Dray.address=<address> \
<classname> <args>
and then the rest of your script should be able to leverage Ray as a distributed framework!
@@ -74,8 +82,6 @@ The most preferable way to run a Ray cluster is via the :ref:`Ray Cluster Launch
This section assumes that you have a list of machines and that the nodes in the cluster can communicate with each other. It also assumes that Ray is installed
on each machine. To install Ray, follow the `installation instructions`_.
To configure the Ray cluster to run Java code, you need to add the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
.. _`installation instructions`: http://docs.ray.io/en/master/installation.html
Starting Ray on each machine
@@ -199,7 +205,7 @@ To run a distributed Ray program, you'll need to execute your program on the sam
.. code-block:: bash
java -classpath /path/to/jars/ \
java -classpath <classpath> \
-Dray.address=<address> \
<classname> <args>
doc/source/cluster/k8s-operator.rst
+234
View File
@@ -0,0 +1,234 @@
.. _k8s-operator:
The Ray Kubernetes Operator
=================================
Ray provides a `Kubernetes Operator`_ for managing autoscaling Ray clusters.
Using the operator provides similar functionality to deploying a Ray cluster using
the :ref:`Ray Cluster Launcher<ref-autoscaling>`. However, working with the operator does not require
running Ray locally -- all interactions with your Ray cluster are mediated by Kubernetes.
The operator makes use of a `Kubernetes Custom Resource`_ called a *RayCluster*.
A RayCluster is specified by a configuration similar to the ``yaml`` files used by the Ray Cluster Launcher.
Internally, the operator uses Ray's autoscaler to manage your Ray cluster. However, the autoscaler runs in a
separate operator pod, rather than on the Ray head node. Applying multiple RayCluster custom resources in the operator's
namespace allows the operator to manage several Ray clusters.
The rest of this document explains step-by-step how to use the Ray Kubernetes Operator to launch a Ray cluster on your existing Kubernetes cluster.
.. role:: bash(code)
:language: bash
.. note::
The example commands in this document launch six Kubernetes pods, using a total of 6 CPU and 3.5Gi memory.
If you are experimenting using a test Kubernetes environment such as `minikube`_, make sure to provision sufficient resources, e.g.
:bash:`minikube start --cpus=6 --memory=\"4G\"`.
Alternatively, reduce resource usage by editing the ``yaml`` files referenced in this document; for example, reduce ``minWorkers``
in ``example_cluster.yaml`` and ``example_cluster2.yaml``.
Applying the RayCluster Custom Resource Definition
--------------------------------------------------
First, we need to apply the `Kubernetes Custom Resource Definition`_ (CRD) defining a RayCluster.
.. note::
Creating a Custom Resource Definition requires the appropriate Kubernetes cluster-level privileges.
.. code-block:: shell
$ kubectl apply -f ray/python/ray/autoscaler/kubernetes/operator_configs/cluster_crd.yaml
customresourcedefinition.apiextensions.k8s.io/rayclusters.cluster.ray.io created
Picking a Kubernetes Namespace
-------------------------------
The rest of the Kubernetes resources we will use are `namespaced`_.
You can use an existing namespace for your Ray clusters or create a new one if you have permissions.
For this example, we will create a namespace called ``ray``.
.. code-block:: shell
$ kubectl create namespace ray
namespace/ray created
Starting the Operator
----------------------
To launch the operator in our namespace, we execute the following command.
.. code-block:: shell
$ kubectl -n ray apply -f ray/python/ray/autoscaler/kubernetes/operator_configs/operator.yaml
serviceaccount/ray-operator-serviceaccount created
role.rbac.authorization.k8s.io/ray-operator-role created
rolebinding.rbac.authorization.k8s.io/ray-operator-rolebinding created
pod/ray-operator-pod created
The output shows that we've launched a Pod named ``ray-operator-pod``. This is the pod that runs the operator process.
The ServiceAccount, Role, and RoleBinding we have created grant the operator pod the `permissions`_ it needs to manage Ray clusters.
Launching Ray Clusters
----------------------
Finally, to launch a Ray cluster, we create a RayCluster custom resource.
.. code-block:: shell
$ kubectl -n ray apply -f ray/python/ray/autoscaler/kubernetes/operator_configs/example_cluster.yaml
raycluster.cluster.ray.io/example-cluster created
The operator detects the RayCluster resource we've created and launches an autoscaling Ray cluster.
Our RayCluster configuration specifies ``minWorkers:2`` in the second entry of ``spec.podTypes``, so we get a head node and two workers upon launch.
.. note::
For more details about RayCluster resources, we recommend take a looking at the annotated example ``example_cluster.yaml`` applied in the last command.
.. code-block:: shell
$ kubectl -n ray get pods
NAME READY STATUS RESTARTS AGE
example-cluster-ray-head-hbxvv 1/1 Running 0 72s
example-cluster-ray-worker-4hvv6 1/1 Running 0 64s
example-cluster-ray-worker-78kp5 1/1 Running 0 64s
ray-operator-pod 1/1 Running 0 2m33s
We see four pods: the operator, the Ray head node, and two Ray worker nodes.
Let's launch another cluster in the same namespace, this one specifiying ``minWorkers:1``.
.. code-block:: shell
$ kubectl -n ray apply -f ray/python/ray/autoscaler/kubernetes/operator_configs/example_cluster2.yaml
We confirm that both clusters are running in our namespace.
.. code-block:: shell
$ kubectl -n ray get rayclusters
NAME AGE
example-cluster 12m
example-cluster2 114s
$ kubectl -n ray get pods
NAME READY STATUS RESTARTS AGE
example-cluster-ray-head-th4wv 1/1 Running 0 10m
example-cluster-ray-worker-q9pjn 1/1 Running 0 10m
example-cluster-ray-worker-qltnp 1/1 Running 0 10m
example-cluster2-ray-head-kj5mg 1/1 Running 0 10s
example-cluster2-ray-worker-qsgnd 1/1 Running 0 1s
ray-operator-pod 1/1 Running 0 10m
Now we can :ref:`run Ray programs<ray-k8s-run>` on our Ray clusters.
Monitoring
----------
Autoscaling logs are written to the operator pod's ``stdout`` and can be accessed with :code:`kubectl logs`.
Each line of output is prefixed by the name of the cluster followed by a colon.
The following command gets the last hundred lines of autoscaling logs for our second cluster.
.. code-block:: shell
$ kubectl -n ray logs ray-operator-pod | grep ^example-cluster2: | tail -n 100
The output should include monitoring updates that look like this:
.. code-block:: shell
example-cluster2:2020-12-12 13:55:36,814 DEBUG autoscaler.py:693 -- Cluster status: 1 nodes
example-cluster2: - MostDelayedHeartbeats: {'172.17.0.4': 0.04093289375305176, '172.17.0.5': 0.04084634780883789}
example-cluster2: - NodeIdleSeconds: Min=36 Mean=38 Max=41
example-cluster2: - ResourceUsage: 0.0/2.0 CPU, 0.0/1.0 Custom1, 0.0/1.0 is_spot, 0.0 GiB/0.58 GiB memory, 0.0 GiB/0.1 GiB object_store_memory
example-cluster2: - TimeSinceLastHeartbeat: Min=0 Mean=0 Max=0
example-cluster2:Worker node types:
example-cluster2: - worker-nodes: 1
example-cluster2:2020-12-12 13:55:36,870 INFO resource_demand_scheduler.py:148 -- Cluster resources: [{'object_store_memory': 1.0, 'node:172.17.0.4': 1.0, 'memory': 5.0, 'CPU': 1.0}, {'object_store_memory': 1.0, 'is_spot': 1.0, 'memory': 6.0, 'node:172.17.0.5': 1.0, 'Custom1': 1.0, 'CPU': 1.0}]
example-cluster2:2020-12-12 13:55:36,870 INFO resource_demand_scheduler.py:149 -- Node counts: defaultdict(<class 'int'>, {'head-node': 1, 'worker-nodes
': 1})
example-cluster2:2020-12-12 13:55:36,870 INFO resource_demand_scheduler.py:159 -- Placement group demands: []
example-cluster2:2020-12-12 13:55:36,870 INFO resource_demand_scheduler.py:186 -- Resource demands: []
example-cluster2:2020-12-12 13:55:36,870 INFO resource_demand_scheduler.py:187 -- Unfulfilled demands: []
example-cluster2:2020-12-12 13:55:36,891 INFO resource_demand_scheduler.py:209 -- Node requests: {}
example-cluster2:2020-12-12 13:55:36,903 DEBUG autoscaler.py:654 -- example-cluster2-ray-worker-tdxdr is not being updated and passes config check (can_update=True).
example-cluster2:2020-12-12 13:55:36,923 DEBUG autoscaler.py:654 -- example-cluster2-ray-worker-tdxdr is not being updated and passes config check (can_update=True).
Updating and Retrying
---------------------
To update a Ray cluster's configuration, edit the ``yaml`` file of the corresponding RayCluster resource
and apply it again:
.. code-block:: shell
$ kubectl -n ray apply -f ray/python/ray/autoscaler/kubernetes/operator_configs/example_cluster.yaml
To force a restart with the same configuration, you can add an `annotation`_ to the RayCluster resource's ``metadata.labels`` field, e.g.
.. code-block:: yaml
apiVersion: cluster.ray.io/v1
kind: RayCluster
metadata:
name: example-cluster
annotations:
try: again
spec:
...
Then reapply the RayCluster, as above.
Currently, editing and reapplying a RayCluster resource will stop and restart Ray processes running on the corresponding
Ray cluster. Similarly, deleting and relaunching the operator pod will stop and restart Ray processes on all Ray clusters in the operator's namespace.
This behavior may be modified in future releases.
Cleaning Up
-----------
We shut down a Ray cluster by deleting the associated RayCluster resource.
Either of the next two commands will delete our second cluster ``example-cluster2``.
.. code-block:: shell
$ kubectl -n ray delete raycluster example-cluster2
# OR
$ kubectl -n ray delete -f ray/python/ray/autoscaler/kubernetes/operator_configs/example_cluster2.yaml
The pods associated with ``example-cluster2`` go into ``TERMINATING`` status. In a few moments, we check that these pods are gone:
.. code-block:: shell
$ kubectl -n ray get pods
NAME READY STATUS RESTARTS AGE
example-cluster-ray-head-th4wv 1/1 Running 0 57m
example-cluster-ray-worker-q9pjn 1/1 Running 0 56m
example-cluster-ray-worker-qltnp 1/1 Running 0 56m
ray-operator-pod 1/1 Running 0 57m
Only the operator pod and the first ``example-cluster`` remain.
To finish clean-up, we delete the cluster ``example-cluster`` and then the operator's resources.
.. code-block:: shell
$ kubectl -n ray delete raycluster example-cluster
$ kubectl -n ray delete -f ray/python/ray/autoscaler/kubernetes/operator_configs/operator.yaml
If you like, you can delete the RayCluster customer resource definition.
(Using the operator again will then require reapplying the CRD.)
.. code-block:: shell
$ kubectl delete crd rayclusters.cluster.ray.io
# OR
$ kubectl delete -f ray/python/ray/autoscaler/kubernetes/operator_configs/cluster_crd.yaml
.. _`Kubernetes Operator`: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
.. _`Kubernetes Custom Resource`: https://kubernetes.io/docs/concepts/extend-kubernetes/api-extension/custom-resources/
.. _`Kubernetes Custom Resource Definition`: https://kubernetes.io/docs/tasks/extend-kubernetes/custom-resources/custom-resource-definitions/
.. _`annotation`: https://kubernetes.io/docs/concepts/overview/working-with-objects/annotations/#attaching-metadata-to-objects
.. _`permissions`: https://kubernetes.io/docs/reference/access-authn-authz/rbac/
.. _`minikube`: https://minikube.sigs.k8s.io/docs/start/
.. _`namespaced`: https://kubernetes.io/docs/concepts/overview/working-with-objects/namespaces/
doc/source/cluster/kubernetes.rst
+10
View File
@@ -12,6 +12,9 @@ This document assumes that you have access to a Kubernetes cluster and have
first walk you through how to deploy a Ray cluster on your existing Kubernetes
cluster, then explore a few different ways to run programs on the Ray cluster.
To learn about deploying an autoscaling Ray cluster using :ref:`Ray's Kubernetes operator<k8s-operator>`, read
:ref:`here<k8s-operator>`.
The configuration ``yaml`` files used here are provided in the `Ray repository`_
as examples to get you started. When deploying real applications, you will probably
want to build and use your own container images, add more worker nodes to the
@@ -38,6 +41,11 @@ flag passed to ``kubectl``.
Starting a Ray Cluster
----------------------
.. toctree::
:hidden:
/cluster/k8s-operator.rst
A Ray cluster consists of a single head node and a set of worker nodes (the
provided ``ray-cluster.yaml`` file will start 3 worker nodes). In the example
Kubernetes configuration, this is implemented as:
@@ -142,6 +150,8 @@ and checking that they are restarted by Kubernetes:
ray-worker-5c49b7cc57-6m4kp 1/1 Running 0 10s
ray-worker-5c49b7cc57-jx2w2 1/1 Running 0 10s
.. _ray-k8s-run:
Running Ray Programs
--------------------
doc/source/conf.py
+3
View File
@@ -94,6 +94,9 @@ for mod_name in MOCK_MODULES:
sys.modules["tensorflow"].VERSION = "9.9.9"
sys.modules["tensorflow.keras.callbacks"] = ChildClassMock()
sys.modules["pytorch_lightning"] = ChildClassMock()
sys.modules["xgboost"] = ChildClassMock()
sys.modules["xgboost.core"] = ChildClassMock()
sys.modules["xgboost.callback"] = ChildClassMock()
class SimpleClass(object):
doc/source/configure.rst
+24 -7
View File
@@ -60,7 +60,9 @@ If using the command line, connect to the Ray cluster as follow:
override this by explicitly setting ``OMP_NUM_THREADS``. ``OMP_NUM_THREADS`` is commonly
used in numpy, PyTorch, and Tensorflow to perform multit-threaded linear algebra.
In multi-worker setting, we want one thread per worker instead of many threads
per worker to avoid contention.
per worker to avoid contention. Some other libraries may have their own way to configure
parallelism. For example, if you're using OpenCV, you should manually set the number of
threads using cv2.setNumThreads(num_threads) (set to 0 to disable multi-threading).
.. _temp-dir-log-files:
@@ -243,24 +245,32 @@ Java Applications
Code Search Path
~~~~~~~~~~~~~~~~
If you want to run a Java application in cluster mode, you must first run ``ray start`` to start the Ray cluster. In addition to any ``ray start`` parameters mentioned above, you must add ``--code-search-path`` to tell Ray where to load jars when starting Java workers. Your jar files must be distributed to all nodes of the Ray cluster before running your code, and this parameter must be set on both the head node and non-head nodes.
If you want to run a Java application in a multi-node cluster, you must specify the code search path in your driver. The code search path is to tell Ray where to load jars when starting Java workers. Your jar files must be distributed to the same path(s) on all nodes of the Ray cluster before running your code.
.. code-block:: bash
$ ray start ... --code-search-path=/path/to/jars
$ java -classpath <classpath> \
-Dray.address=<address> \
-Dray.job.code-search-path=/path/to/jars/ \
<classname> <args>
The ``/path/to/jars`` here points to a directory which contains jars. All jars in the directory will be loaded by workers. You can also provide multiple directories for this parameter.
The ``/path/to/jars/`` here points to a directory which contains jars. All jars in the directory will be loaded by workers. You can also provide multiple directories for this parameter.
.. code-block:: bash
$ ray start ... --code-search-path=/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2
$ java -classpath <classpath> \
-Dray.address=<address> \
-Dray.job.code-search-path=/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2 \
<classname> <args>
Code search path is also used for loading Python code if it's specified. This is required for :ref:`cross_language`. If code search path is specified, you can only run Python remote functions which can be found in the code search path.
You don't need to configure code search path if you run a Java application in a single-node cluster.
You don't need to configure code search path if you run a Java application in single machine mode.
See ``ray.job.code-search-path`` under :ref:`Driver Options <java-driver-options>` for more information.
.. note:: Currently we don't provide a way to configure Ray when running a Java application in single machine mode. If you need to configure Ray, run ``ray start`` to start the Ray cluster first.
.. _java-driver-options:
Driver Options
~~~~~~~~~~~~~~
@@ -287,4 +297,11 @@ The list of available driver options:
- Type: ``Boolean``
- Default: ``false``
- ``ray.job.code-search-path``
- The paths for Java workers to load code from. Currently only directories are supported. You can specify one or more directories split by a ``:``. You don't need to configure code search path if you run a Java application in single machine mode or local mode. Code search path is also used for loading Python code if it's specified. This is required for :ref:`cross_language`. If code search path is specified, you can only run Python remote functions which can be found in the code search path.
- Type: ``String``
- Default: empty string.
- Example: ``/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2``
.. _`Apache Arrow`: https://arrow.apache.org/
doc/source/cross-language.rst
+32 -6
View File
@@ -5,20 +5,46 @@ Cross-language programming
This page will show you how to use Ray's cross-language programming feature.
Setup the cluster
Setup the driver
-----------------
We need to set the ``--code-search-path`` option on ``ray start`` command. See :ref:`code_search_path` for more details.
We need to set :ref:`code_search_path` in your driver.
.. code-block:: bash
.. tabs::
ray start ... --code-search-path=/path/to/code
.. group-tab:: Python
.. code-block:: python
ray.init(job_config=ray.job_config.JobConfig(code_search_path="/path/to/code"))
.. group-tab:: Java
.. code-block:: bash
java -classpath <classpath> \
-Dray.address=<address> \
-Dray.job.code-search-path=/path/to/code/ \
<classname> <args>
You may want to include multiple directories to load both Python and Java code for workers, if they are placed in different directories.
.. code-block:: bash
.. tabs::
ray start ... --code-search-path=/path/to/jars:/path/to/pys
.. group-tab:: Python
.. code-block:: python
ray.init(job_config=ray.job_config.JobConfig(code_search_path="/path/to/jars:/path/to/pys"))
.. group-tab:: Java
.. code-block:: bash
java -classpath <classpath> \
-Dray.address=<address> \
-Dray.job.code-search-path=/path/to/jars:/path/to/pys \
<classname> <args>
Python calling Java
-------------------
doc/source/development.rst
+1 -1
View File
@@ -23,7 +23,7 @@ RLlib, Tune, Autoscaler, and most Python files do not require you to build and c
.. code-block:: shell
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp38-cp38-manylinux2014_x86_64.whl
pip install -U https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp38-cp38-manylinux2014_x86_64.whl
2. Fork and clone the project to your machine. Connect your repository to the upstream (main project) ray repository.
doc/source/getting-involved.rst
+5 -2
View File
@@ -6,7 +6,10 @@ Getting Involved / Contributing
Ray is more than a framework for distributed applications but also an active community of developers,
researchers, and folks that love machine learning.
.. tip:: Join our `community slack <https://forms.gle/9TSdDYUgxYs8SA9e8>`_ to discuss Ray! The community is extremely active in helping people succeed in building their ray applications.
.. tip:: Join our `community Slack <https://forms.gle/9TSdDYUgxYs8SA9e8>`_ to
discuss Ray or ask questions on `our forum <https://discuss.ray.io/>`_! The
community is extremely active in helping people succeed in building their
Ray applications.
You can join (and Star!) us on `on GitHub`_.
@@ -141,7 +144,7 @@ You can run the following locally:
.. code-block:: shell
ray/scripts/format.sh
./ci/travis/format.sh
An output like the following indicates failure:
doc/source/installation.rst
+82 -20
View File
@@ -15,7 +15,7 @@ You can install the latest official version of Ray as follows. Official releases
.. code-block:: bash
pip install -U ray # also recommended: ray[debug]
pip install -U ray
**Note for Windows Users:** To use Ray on Windows, Visual C++ runtime must be installed (see :ref:`Windows Dependencies <windows-dependencies>` section). If you run into any issues, please see the :ref:`Windows Support <windows-support>` section.
@@ -55,20 +55,21 @@ instead of the ones above:
`Linux Python 3.6`_ `MacOS Python 3.6`_ `Windows Python 3.6`_
=================== =================== ======================
.. _`Linux Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp39-cp39-manylinux2014_x86_64.whl
.. _`Linux Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp38-cp38-manylinux2014_x86_64.whl
.. _`Linux Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp37-cp37m-manylinux2014_x86_64.whl
.. _`Linux Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp36-cp36m-manylinux2014_x86_64.whl
.. _`MacOS Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp39-cp39-macosx_10_13_x86_64.whl
.. _`MacOS Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp38-cp38-macosx_10_13_x86_64.whl
.. _`MacOS Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp37-cp37m-macosx_10_13_intel.whl
.. _`MacOS Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp36-cp36m-macosx_10_13_intel.whl
.. _`Linux Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp39-cp39-manylinux2014_x86_64.whl
.. _`Linux Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp38-cp38-manylinux2014_x86_64.whl
.. _`Linux Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp37-cp37m-manylinux2014_x86_64.whl
.. _`Linux Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp36-cp36m-manylinux2014_x86_64.whl
.. _`Windows Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp39-cp39-win_amd64.whl
.. _`Windows Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp38-cp38-win_amd64.whl
.. _`Windows Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp37-cp37m-win_amd64.whl
.. _`Windows Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.1.0.dev0-cp36-cp36m-win_amd64.whl
.. _`MacOS Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp39-cp39-macosx_10_13_x86_64.whl
.. _`MacOS Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp38-cp38-macosx_10_13_x86_64.whl
.. _`MacOS Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp37-cp37m-macosx_10_13_intel.whl
.. _`MacOS Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp36-cp36m-macosx_10_13_intel.whl
.. _`Windows Python 3.9`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp39-cp39-win_amd64.whl
.. _`Windows Python 3.8`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp38-cp38-win_amd64.whl
.. _`Windows Python 3.7`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp37-cp37m-win_amd64.whl
.. _`Windows Python 3.6`: https://s3-us-west-2.amazonaws.com/ray-wheels/latest/ray-1.2.0.dev0-cp36-cp36m-win_amd64.whl
Installing from a specific commit
@@ -80,11 +81,11 @@ You can install the Ray wheels of any particular commit on ``master`` with the f
pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/{COMMIT_HASH}/ray-{RAY_VERSION}-{PYTHON_VERSION}-{PYTHON_VERSION}m-{OS_VERSION}_intel.whl
For example, here are the Ray 1.1.0.dev0 wheels for Python 3.5, MacOS for commit ``a0ba4499ac645c9d3e82e68f3a281e48ad57f873``:
For example, here are the Ray 1.2.0.dev0 wheels for Python 3.5, MacOS for commit ``a0ba4499ac645c9d3e82e68f3a281e48ad57f873``:
.. code-block:: bash
pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/a0ba4499ac645c9d3e82e68f3a281e48ad57f873/ray-1.1.0.dev0-cp35-cp35m-macosx_10_13_intel.whl
pip install https://ray-wheels.s3-us-west-2.amazonaws.com/master/a0ba4499ac645c9d3e82e68f3a281e48ad57f873/ray-1.2.0.dev0-cp35-cp35m-macosx_10_13_intel.whl
.. _ray-install-java:
@@ -139,7 +140,7 @@ The latest Ray Java snapshot can be found in `sonatype repository <https://oss.s
.. note::
When you run ``pip install`` to install Ray, Java jars are installed as well. The above dependencies are only used to build your Java code and to run your code in local or single machine mode.
When you run ``pip install`` to install Ray, Java jars are installed as well. The above dependencies are only used to build your Java code and to run your code in local mode.
If you want to run your Java code in a multi-node Ray cluster, it's better to exclude Ray jars when packaging your code to avoid jar conficts if the versions (installed Ray with ``pip install`` and maven dependencies) don't match.
@@ -237,13 +238,45 @@ However, should you need to build from source, follow :ref:`these instructions f
Docker Source Images
--------------------
Most users should pull a Docker image from the Ray Docker Hub.
Most users should pull a Docker image from the `Ray Docker Hub. <https://hub.docker.com/r/rayproject/>`_
- The ``rayproject/ray`` image has ray and all required dependencies. It comes with anaconda and Python 3.7.
- The ``rayproject/autoscaler`` image has the above features as well as many additional libraries.
- The ``rayproject/ray`` `image has ray and all required dependencies. It comes with anaconda and Python 3.7. <https://hub.docker.com/r/rayproject/ray>`_
- The ``rayproject/ray-ml`` `image has the above features as well as many additional libraries. <https://hub.docker.com/r/rayproject/ray-ml>`_
- The ``rayproject/base-deps`` and ``rayproject/ray-deps`` are for the linux and python dependencies respectively.
These images are tagged by their release number (or commit hash for nightlies) as well as a ``"-gpu"`` if they are GPU compatible.
Image releases are `tagged` using the following format:
.. list-table::
:widths: 25 50
:header-rows: 1
* - Tag
- Description
* - latest
- The most recent Ray release.
* - 1.x.x
- A specific Ray release.
* - nightly
- The most recent Ray build (the most recent commit on Github ``master``)
* - Git SHA
- A specific nightly build (uses a SHA from the Github ``master``).
Each tag has `variants` that add or change functionality:
.. list-table::
:widths: 16 40
:header-rows: 1
* - Variant
- Description
* - -gpu
- These are based off of an NVIDIA CUDA image. They require the Nvidia Docker Runtime.
* - -cpu
- These are based off of an Ubuntu image.
* - <no tag>
- Aliases to ``-cpu`` tagged images
If you want to tweak some aspect of these images and build them locally, refer to the following script:
@@ -308,3 +341,32 @@ that you've cloned the git repository.
.. code-block:: bash
python -m pytest -v python/ray/tests/test_mini.py
Troubleshooting
---------------
If importing Ray (``python3 -c "import ray"``) in your development clone results
in this error:
.. code-block:: python
Traceback (most recent call last):
File "<string>", line 1, in <module>
File ".../ray/python/ray/__init__.py", line 63, in <module>
import ray._raylet # noqa: E402
File "python/ray/_raylet.pyx", line 98, in init ray._raylet
import ray.memory_monitor as memory_monitor
File ".../ray/python/ray/memory_monitor.py", line 9, in <module>
import psutil # noqa E402
File ".../ray/python/ray/thirdparty_files/psutil/__init__.py", line 159, in <module>
from . import _psosx as _psplatform
File ".../ray/python/ray/thirdparty_files/psutil/_psosx.py", line 15, in <module>
from . import _psutil_osx as cext
ImportError: cannot import name '_psutil_osx' from partially initialized module 'psutil' (most likely due to a circular import) (.../ray/python/ray/thirdparty_files/psutil/__init__.py)
Then you should run the following commands:
.. code-block:: bash
rm -rf python/ray/thirdparty_files/
python3 -m pip install setproctitle
doc/source/placement-group.rst
-1
View File
@@ -124,7 +124,6 @@ Let's create a placement group. Recall that each bundle is a collection of resou
- "CPU" will correspond with `num_cpus` as used in `ray.remote`
- "GPU" will correspond with `num_gpus` as used in `ray.remote`
- "MEM" will correspond with `memory` as used in `ray.remote`
- Other resources will correspond with `resources` as used in `ray.remote`.
Once the placement group reserves resources, original resources are unavailable until the placement group is removed. For example:
doc/source/ray-overview/index.rst
+1 -1
View File
@@ -196,7 +196,7 @@ RLlib Quick Start
.. code-block:: bash
pip install tensorflow # or tensorflow-gpu
pip install ray[rllib] # also recommended: ray[debug]
pip install ray[rllib]
.. code-block:: python
doc/source/ray-overview/involvement.rst
+1 -1
View File
@@ -1,7 +1,7 @@
Ray is more than a framework for distributed applications but also an active community of developers,
researchers, and folks that love machine learning. Here's a list of tips for getting involved with the Ray community:
- Join our `community slack <https://forms.gle/9TSdDYUgxYs8SA9e8>`_ to discuss Ray!
- Join our `community Slack <https://forms.gle/9TSdDYUgxYs8SA9e8>`_ to discuss Ray!
- Star and follow us on `on GitHub`_.
- To post questions or feature requests, check out the `Discussion Board`_!
- Follow us and spread the word on `Twitter`_!
doc/source/rllib.rst
+1 -1
View File
@@ -23,7 +23,7 @@ RLlib has extra dependencies on top of ``ray``. First, you'll need to install ei
.. code-block:: bash
pip install 'ray[rllib]' # also recommended: ray[debug]
pip install 'ray[rllib]'
Then, you can try out training in the following equivalent ways:
doc/source/serve/advanced.rst
+17 -5
View File
@@ -81,6 +81,12 @@ If you *do* want to enable this parallelism in your Serve backend, just set OMP_
client.create_backend("parallel_backend", MyBackend, 12)
.. note::
Some other libraries may not respect ``OMP_NUM_THREADS`` and have their own way to configure parallelism.
For example, if you're using OpenCV, you'll need to manually set the number of threads using ``cv2.setNumThreads(num_threads)`` (set to 0 to disable multi-threading).
You can check the configuration using ``cv2.getNumThreads()`` and ``cv2.getNumberOfCPUs()``.
.. _serve-batching:
Batching to improve performance
@@ -306,12 +312,18 @@ and another named ``ray-tf2`` with Ray Serve and Tensorflow 2. The Ray and
python versions must be the same in both environments. To specify
an environment for a backend to use, simply pass the environment name in to
:mod:`client.create_backend <ray.serve.api.Client.create_backend>`
as shown below. Be sure to run the script in an activated conda environment
(not required to be ``ray-tf1`` or ``ray-tf2``).
as shown below.
.. literalinclude:: ../../../python/ray/serve/examples/doc/conda_env.py
Alternatively, you may omit the argument ``env`` and call
:mod:`client.create_backend <ray.serve.api.Client.create_backend>`
from a script running in the conda environment you want the backend to run in.
.. warning::
The script must be run in an activated conda environment (not required to be
``ray-tf1`` or ``ray-tf2``). We hope to remove this restriction in the
future.
.. note::
If the argument ``env`` is omitted, backends will be started in the same
conda environment as the caller of
:mod:`client.create_backend <ray.serve.api.Client.create_backend>` by
default.
doc/source/serve/index.rst
+1 -1
View File
@@ -37,7 +37,7 @@ Since Serve is built on Ray, it also allows you to scale to many machines, in yo
Installation
============
Ray Serve supports Python versions 3.6 and higher. To install Ray Serve:
Ray Serve supports Python versions 3.6 through 3.8. To install Ray Serve:
.. code-block:: bash
doc/source/serve/key-concepts.rst
+5 -1
View File
@@ -19,7 +19,11 @@ Backends
Backends define the implementation of your business logic or models that will handle requests when queries come in to :ref:`serve-endpoint`.
In order to support seamless scalability backends can have many replicas, which are individual processes running in the Ray cluster to handle requests.
To define a backend, first you must define the "handler" or the business logic you'd like to respond with.
The handler should take as input a `Flask Request object <https://flask.palletsprojects.com/en/1.1.x/api/?highlight=request#flask.Request>`_ and return any JSON-serializable object as output.
The handler should take as input a `Flask Request object <https://flask.palletsprojects.com/en/1.1.x/api/?highlight=request#flask.Request>`_.
The handler should return any JSON-serializable object as output. For a more customizable response type, the handler may return a
`Starlette Response object <https://www.starlette.io/responses/>`_.
In the future, Ray Serve will support `Starlette Request objects <https://www.starlette.io/requests/>`_ as input as well.
A backend is defined using :mod:`client.create_backend <ray.serve.api.Client.create_backend>`, and the implementation can be defined as either a function or a class.
Use a function when your response is stateless and a class when you might need to maintain some state (like a model).
When using a class, you can specify arguments to be passed to the constructor in :mod:`client.create_backend <ray.serve.api.Client.create_backend>`, shown below.
doc/source/starting-ray.rst
+19 -10
View File
@@ -125,25 +125,34 @@ Use ``ray start`` from the CLI to start a 1 node ray runtime on a machine. This
...
You can connect to this Ray runtime by starting a driver process on the same node as where you ran ``ray start``:
.. tabs::
.. group-tab:: python
.. code-tab:: python
You can connect to this Ray runtime by starting a Python process that calls the following on the same node as where you ran ``ray start``:
.. code-block:: python
# This must
import ray
ray.init(address='auto')
# This must
import ray
ray.init(address='auto')
.. group-tab:: java
.. code-block:: java
If you want to run Java code, you need to specify the classpath via the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
import io.ray.api.Ray;
public class MyRayApp {
public static void main(String[] args) {
Ray.init();
...
}
}
.. code-block:: bash
$ ray start ... --code-search-path=/path/to/jars
java -classpath <classpath> \
-Dray.address=<address> \
<classname> <args>
You can connect other nodes to the head node, creating a Ray cluster by also calling ``ray start`` on those nodes. See :ref:`manual-cluster` for more details. Calling ``ray.init(address="auto")`` on any of the cluster machines will connect to the ray cluster.
doc/source/tune/api_docs/execution.rst
-8
View File
@@ -23,14 +23,6 @@ tune.with_parameters
.. autofunction:: ray.tune.with_parameters
.. _tune-stop-ref:
Stopper (tune.Stopper)
----------------------
.. autoclass:: ray.tune.Stopper
:members: __call__, stop_all
.. _tune-sync-config:
tune.SyncConfig
doc/source/tune/api_docs/overview.rst
+1
View File
@@ -21,6 +21,7 @@ on `Github`_.
suggestion.rst
schedulers.rst
sklearn.rst
stoppers.rst
logging.rst
integration.rst
internals.rst
doc/source/tune/api_docs/stoppers.rst
+46
View File
@@ -0,0 +1,46 @@
.. _tune-stoppers:
Stopping mechanisms (tune.stopper)
==================================
In addition to Trial Schedulers like :ref:`ASHA <tune-scheduler-hyperband>`, where a number of
trials are stopped if they perform subpar, Ray Tune also supports custom stopping mechanisms to stop trials early. For instance, stopping mechanisms can specify to stop trials when they reached a plateau and the metric
doesn't change anymore.
Ray Tune comes with several stopping mechanisms out of the box. For custom stopping behavior, you can
inherit from the :class:`Stopper <ray.tune.Stopper>` class.
Other stopping behaviors are described :ref:`in the user guide <tune-stopping>`.
.. contents::
:local:
:depth: 1
.. _tune-stop-ref:
Stopper (tune.Stopper)
----------------------
.. autoclass:: ray.tune.Stopper
:members: __call__, stop_all
MaximumIterationStopper (tune.stopper.MaximumIterationStopper)
--------------------------------------------------------------
.. autoclass:: ray.tune.stopper.MaximumIterationStopper
ExperimentPlateauStopper (tune.stopper.ExperimentPlateauStopper)
----------------------------------------------------------------
.. autoclass:: ray.tune.stopper.ExperimentPlateauStopper
TrialPlateauStopper (tune.stopper.TrialPlateauStopper)
------------------------------------------------------
.. autoclass:: ray.tune.stopper.TrialPlateauStopper
TimeoutStopper (tune.stopper.TimeoutStopper)
--------------------------------------------
.. autoclass:: ray.tune.stopper.TimeoutStopper
doc/source/tune/user-guide.rst
+7 -3
View File
@@ -305,7 +305,9 @@ and passed to your trainable as a parameter.
Stopping Trials
---------------
You can control when trials are stopped early by passing the ``stop`` argument to ``tune.run``. This argument takes either a dictionary or a function.
You can control when trials are stopped early by passing the ``stop`` argument to ``tune.run``.
This argument takes, a dictionary, a function, or a :class:`Stopper <ray.tune.stopper.Stopper>` class
as an argument.
If a dictionary is passed in, the keys may be any field in the return result of ``tune.report`` in the Function API or ``step()`` (including the results from ``step`` and auto-filled metrics).
@@ -329,7 +331,7 @@ For more flexibility, you can pass in a function instead. If a function is passe
tune.run(my_trainable, stop=stopper)
Finally, you can implement the ``Stopper`` abstract class for stopping entire experiments. For example, the following example stops all trials after the criteria is fulfilled by any individual trial, and prevents new ones from starting:
Finally, you can implement the :class:`Stopper <ray.tune.stopper.Stopper>` abstract class for stopping entire experiments. For example, the following example stops all trials after the criteria is fulfilled by any individual trial, and prevents new ones from starting:
.. code-block:: python
@@ -352,7 +354,9 @@ Finally, you can implement the ``Stopper`` abstract class for stopping entire ex
tune.run(my_trainable, stop=stopper)
Note that in the above example the currently running trials will not stop immediately but will do so once their current iterations are complete. See the :ref:`tune-stop-ref` documentation.
Note that in the above example the currently running trials will not stop immediately but will do so once their current iterations are complete.
Ray Tune comes with a set of out-of-the-box stopper classes. See the :ref:`Stopper <tune-stoppers>` documentation.
.. _tune-logging:
doc/source/xgboost-ray.rst
-18
View File
@@ -143,21 +143,3 @@ the `examples folder <https://github.com/ray-project/xgboost_ray/tree/master/exa
* `[download dataset (2.6 GB)] <https://archive.ics.uci.edu/ml/machine-learning-databases/00280/HIGGS.csv.gz>`__
* `HIGGS classification example with Parquet <https://github.com/ray-project/xgboost_ray/tree/master/examples/higgs_parquet.py>`__ (uses the same dataset)
* `Test data classification <https://github.com/ray-project/xgboost_ray/tree/master/examples/train_on_test_data.py>`__ (uses a self-generated dataset)
Package Reference
-----------------
Training/Validation
~~~~~~~~~~~~~~~~~~~
.. autoclass:: ray.util.xgboost.RayParams
.. autofunction:: ray.util.xgboost.train
.. autofunction:: ray.util.xgboost.predict
RayDMatrix
~~~~~~~~~~
.. autoclass:: ray.util.xgboost.RayDMatrix