mirror of
https://github.com/wassname/ray.git
synced 2026-08-12 12:20:11 +08:00
[Multi-tenancy] Delete flag enable_multi_tenancy and remove old code path (#10573)
This commit is contained in:
Kai Yang
@@ -21,26 +21,34 @@ Clusters are started with the :ref:`Ray Cluster Launcher <ref-automatic-cluster>
|
||||
|
||||
You can also create a Ray cluster using a standard cluster manager such as :ref:`Kubernetes <ray-k8s-deploy>`, :ref:`YARN <ray-yarn-deploy>`, or :ref:`SLURM <ray-slurm-deploy>`.
|
||||
|
||||
After a cluster is started, you need to connect your program to the Ray cluster.
|
||||
After a cluster is started, you need to connect your program to the Ray cluster by starting a driver process on the same node as where you ran ``ray start``:
|
||||
|
||||
.. tabs::
|
||||
.. group-tab:: python
|
||||
.. code-tab:: python
|
||||
|
||||
You can connect to this Ray runtime by starting a Python process that calls the following on the same node as where you ran ``ray start``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# This must
|
||||
import ray
|
||||
ray.init(address='auto')
|
||||
# This must
|
||||
import ray
|
||||
ray.init(address='auto')
|
||||
|
||||
.. group-tab:: java
|
||||
|
||||
If you want to run Java code, you need to specify the classpath via the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
|
||||
.. code-block:: java
|
||||
|
||||
import io.ray.api.Ray;
|
||||
|
||||
public class MyRayApp {
|
||||
|
||||
public static void main(String[] args) {
|
||||
Ray.init();
|
||||
...
|
||||
}
|
||||
}
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ray start ... --code-search-path=/path/to/jars
|
||||
java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
<classname> <args>
|
||||
|
||||
and then the rest of your script should be able to leverage Ray as a distributed framework!
|
||||
|
||||
@@ -74,8 +82,6 @@ The most preferable way to run a Ray cluster is via the :ref:`Ray Cluster Launch
|
||||
This section assumes that you have a list of machines and that the nodes in the cluster can communicate with each other. It also assumes that Ray is installed
|
||||
on each machine. To install Ray, follow the `installation instructions`_.
|
||||
|
||||
To configure the Ray cluster to run Java code, you need to add the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
|
||||
|
||||
.. _`installation instructions`: http://docs.ray.io/en/master/installation.html
|
||||
|
||||
Starting Ray on each machine
|
||||
@@ -199,7 +205,7 @@ To run a distributed Ray program, you'll need to execute your program on the sam
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
java -classpath /path/to/jars/ \
|
||||
java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
<classname> <args>
|
||||
|
||||
|
||||
@@ -245,24 +245,32 @@ Java Applications
|
||||
Code Search Path
|
||||
~~~~~~~~~~~~~~~~
|
||||
|
||||
If you want to run a Java application in cluster mode, you must first run ``ray start`` to start the Ray cluster. In addition to any ``ray start`` parameters mentioned above, you must add ``--code-search-path`` to tell Ray where to load jars when starting Java workers. Your jar files must be distributed to all nodes of the Ray cluster before running your code, and this parameter must be set on both the head node and non-head nodes.
|
||||
If you want to run a Java application in a multi-node cluster, you must specify the code search path in your driver. The code search path is to tell Ray where to load jars when starting Java workers. Your jar files must be distributed to the same path(s) on all nodes of the Ray cluster before running your code.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ray start ... --code-search-path=/path/to/jars
|
||||
$ java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
-Dray.job.code-search-path=/path/to/jars/ \
|
||||
<classname> <args>
|
||||
|
||||
The ``/path/to/jars`` here points to a directory which contains jars. All jars in the directory will be loaded by workers. You can also provide multiple directories for this parameter.
|
||||
The ``/path/to/jars/`` here points to a directory which contains jars. All jars in the directory will be loaded by workers. You can also provide multiple directories for this parameter.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ray start ... --code-search-path=/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2
|
||||
$ java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
-Dray.job.code-search-path=/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2 \
|
||||
<classname> <args>
|
||||
|
||||
Code search path is also used for loading Python code if it's specified. This is required for :ref:`cross_language`. If code search path is specified, you can only run Python remote functions which can be found in the code search path.
|
||||
You don't need to configure code search path if you run a Java application in a single-node cluster.
|
||||
|
||||
You don't need to configure code search path if you run a Java application in single machine mode.
|
||||
See ``ray.job.code-search-path`` under :ref:`Driver Options <java-driver-options>` for more information.
|
||||
|
||||
.. note:: Currently we don't provide a way to configure Ray when running a Java application in single machine mode. If you need to configure Ray, run ``ray start`` to start the Ray cluster first.
|
||||
|
||||
.. _java-driver-options:
|
||||
|
||||
Driver Options
|
||||
~~~~~~~~~~~~~~
|
||||
|
||||
@@ -289,4 +297,11 @@ The list of available driver options:
|
||||
- Type: ``Boolean``
|
||||
- Default: ``false``
|
||||
|
||||
- ``ray.job.code-search-path``
|
||||
|
||||
- The paths for Java workers to load code from. Currently only directories are supported. You can specify one or more directories split by a ``:``. You don't need to configure code search path if you run a Java application in single machine mode or local mode. Code search path is also used for loading Python code if it's specified. This is required for :ref:`cross_language`. If code search path is specified, you can only run Python remote functions which can be found in the code search path.
|
||||
- Type: ``String``
|
||||
- Default: empty string.
|
||||
- Example: ``/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2``
|
||||
|
||||
.. _`Apache Arrow`: https://arrow.apache.org/
|
||||
|
||||
@@ -5,20 +5,46 @@ Cross-language programming
|
||||
|
||||
This page will show you how to use Ray's cross-language programming feature.
|
||||
|
||||
Setup the cluster
|
||||
Setup the driver
|
||||
-----------------
|
||||
|
||||
We need to set the ``--code-search-path`` option on ``ray start`` command. See :ref:`code_search_path` for more details.
|
||||
We need to set :ref:`code_search_path` in your driver.
|
||||
|
||||
.. code-block:: bash
|
||||
.. tabs::
|
||||
|
||||
ray start ... --code-search-path=/path/to/code
|
||||
.. group-tab:: Python
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ray.init(job_config=ray.job_config.JobConfig(code_search_path="/path/to/code"))
|
||||
|
||||
.. group-tab:: Java
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
-Dray.job.code-search-path=/path/to/code/ \
|
||||
<classname> <args>
|
||||
|
||||
You may want to include multiple directories to load both Python and Java code for workers, if they are placed in different directories.
|
||||
|
||||
.. code-block:: bash
|
||||
.. tabs::
|
||||
|
||||
ray start ... --code-search-path=/path/to/jars:/path/to/pys
|
||||
.. group-tab:: Python
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
ray.init(job_config=ray.job_config.JobConfig(code_search_path="/path/to/jars:/path/to/pys"))
|
||||
|
||||
.. group-tab:: Java
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
-Dray.job.code-search-path=/path/to/jars:/path/to/pys \
|
||||
<classname> <args>
|
||||
|
||||
Python calling Java
|
||||
-------------------
|
||||
|
||||
+19
-10
@@ -125,25 +125,34 @@ Use ``ray start`` from the CLI to start a 1 node ray runtime on a machine. This
|
||||
...
|
||||
|
||||
|
||||
You can connect to this Ray runtime by starting a driver process on the same node as where you ran ``ray start``:
|
||||
|
||||
.. tabs::
|
||||
.. group-tab:: python
|
||||
.. code-tab:: python
|
||||
|
||||
You can connect to this Ray runtime by starting a Python process that calls the following on the same node as where you ran ``ray start``:
|
||||
|
||||
.. code-block:: python
|
||||
|
||||
# This must
|
||||
import ray
|
||||
ray.init(address='auto')
|
||||
# This must
|
||||
import ray
|
||||
ray.init(address='auto')
|
||||
|
||||
.. group-tab:: java
|
||||
|
||||
.. code-block:: java
|
||||
|
||||
If you want to run Java code, you need to specify the classpath via the ``--code-search-path`` option. See :ref:`code_search_path` for more details.
|
||||
import io.ray.api.Ray;
|
||||
|
||||
public class MyRayApp {
|
||||
|
||||
public static void main(String[] args) {
|
||||
Ray.init();
|
||||
...
|
||||
}
|
||||
}
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ray start ... --code-search-path=/path/to/jars
|
||||
java -classpath <classpath> \
|
||||
-Dray.address=<address> \
|
||||
<classname> <args>
|
||||
|
||||
|
||||
You can connect other nodes to the head node, creating a Ray cluster by also calling ``ray start`` on those nodes. See :ref:`manual-cluster` for more details. Calling ``ray.init(address="auto")`` on any of the cluster machines will connect to the ray cluster.
|
||||
|
||||
Reference in New Issue
Block a user