[Core] Allow users to specify the classpath and import path (#10560)

* move job resource path to job config

* job resource path support list

* job resource path support for python

* fix job_resource_path support

* fix worker command

* fix job config

* use jar file instead of parent path

* fix job resource path

* add test to test.sh

* lint

* Update java/runtime/src/main/resources/ray.default.conf

Co-authored-by: Kai Yang <kfstorm@outlook.com>

* fix testGetFunctionFromLocalResource

* lint

* fix rebase

* add jars in resource path to classloader

* add job_resource_path to worker

* add ray stop

* rename job_resource_path to resource_path

* fix resource_path

* refine resource_path comments

* rename job resource path to code search path

* Add instruction about starting a cross-language cluster

* fix ClassLoaderTest.java

* add code-search-path to RunManager

* refine comments for code-search-path

* rename resourcePath to codeSearchPath

* Update doc

* fix

* rename resourcePath to codeSearchPath

* update doc

* filter out empty path

* fix comments

* fix comments

* fix tests

* revert pom

* lint

* fix doc

* update doc

* Apply suggestions from code review

* lint

Co-authored-by: Kai Yang <kfstorm@outlook.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>

doc/source/cross-language.rst

@@ -3,6 +3,25 @@ Cross-language programming
 
 This page will show you how to use Ray's cross-language programming feature.
 
+Setup the cluster
+-----------------
+
+We need to set the ``--code-search-path`` option on ``ray start`` command. The ``--code-search-path`` option instructs workers to load Java or Python code from the specified code search path.
+
+.. code-block:: bash
+
+  ray start ... --code-search-path=/path/to/code
+
+You can also provide multiple directories for this option.
+
+.. code-block:: bash
+
+  ray start ... --code-search-path=/path/to/jars1:/path/to/jars2:/path/to/pys1:/path/to/pys2
+
+.. note:
+
+  If ``--code-search-path`` is specified, you can only run remote functions which can be found in ``--code-search-path``.
+
 Python calling Java
 -------------------
 
@@ -41,7 +60,7 @@ from the above Java class.
 
   import ray
 
-  ray.init(_load_code_from_local=True)
+  ray.init(address="auto")
 
   # Define a Java class.
   counter_class = ray.java_actor_class(
@@ -57,13 +76,13 @@ from the above Java class.
   # Define a Java function.
   add_function = ray.java_function(
         "io.ray.demo.Math", "add")
-  
+
   # Call the Java remote function.
   obj_ref3 = add_function.remote(1, 2)
   assert ray.get(obj_ref3) == 3
 
   ray.shutdown()
-  
+
 Java calling Python
 -------------------
 
@@ -92,7 +111,7 @@ Suppose we have a Python module as follows:
 
   * The function or class should be decorated by `@ray.remote`.
 
-Then, in Java, we can call the above Python remote function, or create an actor 
+Then, in Java, we can call the above Python remote function, or create an actor
 from the above Python class.
 
 .. code-block:: java
@@ -139,7 +158,7 @@ Cross-language data serialization
 
 The arguments and return values of ray call can be serialized & deserialized
 automatically if their types are the following:
-  
+
   - Primitive data types
       ===========   =======  =======
       MessagePack   Python   Java
@@ -168,7 +187,7 @@ automatically if their types are the following:
     float type to receive the input argument, the double precision Python data
     will be reduced to float precision in Java.
   * BigInteger can support max value of 2^64-1, please refer to:
-    https://github.com/msgpack/msgpack/blob/master/spec.md#int-format-family. 
+    https://github.com/msgpack/msgpack/blob/master/spec.md#int-format-family.
     If the value larger than 2^64-1, then transfer the BigInteger:
 
       - From Java to Python: *raise an exception*
@@ -272,7 +291,7 @@ Then, run the following code:
 
   import ray
 
-  ray.init(_load_code_from_local=True)
+  ray.init(address="auto")
 
   obj_ref = ray.java_function(
         "io.ray.demo.MyRayClass",

doc/source/starting-ray.rst

@@ -135,6 +135,8 @@ There are two steps needed to use Ray in a distributed setting:
 
           ray up cluster.yaml
 
+      To configure the Ray cluster to run Java code, you need to add the ``--code-search-path`` option. It's used to specify classpath for workers in the cluster. Your jar files must be distributed to all the nodes of the Ray cluster before running your code. You also need to make sure the paths of jar files are the same among nodes.
+
       You can monitor the Ray cluster status with ``ray monitor cluster.yaml`` and ssh into the head node with ``ray attach cluster.yaml``.
 
     2. Specify the address of the Ray cluster when initializing Ray in your code. This causes Ray to connect to the existing cluster instead of starting a new one on the local node.
@@ -152,14 +154,11 @@ There are two steps needed to use Ray in a distributed setting:
 
           You need to add the ``ray.redis.address`` parameter to your command line (like ``-Dray.redis.address=...``).
 
-          You need to add the ``ray.job.resource-path`` parameter as well. Your jar files must be distributed to all the nodes of the Ray cluster before running your code. You also need to make sure the paths of jar files are the same among nodes. Let's say your jar files are located in ``/path/to/jars/``, all files under this path will be loaded by worker processes.
-
           To connect your program to the Ray cluster, run it like this:
 
               .. code-block:: bash
 
                   java -classpath /path/to/jars/ \
-                    -Dray.job.resource-path=/path/to/jars/ \
                     -Dray.redis.address=<address> \
                     <classname> <args>
 
@@ -189,6 +188,11 @@ The command will print out the address of the Redis server that was started (and
 
     $ ray start --address=<address>
 
+If you want to run Java code, you need to specify the classpath via the ``--code-search-path`` option.
+
+.. code-block:: bash
+
+  $ ray start ... --code-search-path=/path/to/jars
 
 Local mode
 ----------