[Core] Allow users to specify the classpath and import path (#10560)

* move job resource path to job config

* job resource path support list

* job resource path support for python

* fix job_resource_path support

* fix worker command

* fix job config

* use jar file instead of parent path

* fix job resource path

* add test to test.sh

* lint

* Update java/runtime/src/main/resources/ray.default.conf

Co-authored-by: Kai Yang <kfstorm@outlook.com>

* fix testGetFunctionFromLocalResource

* lint

* fix rebase

* add jars in resource path to classloader

* add job_resource_path to worker

* add ray stop

* rename job_resource_path to resource_path

* fix resource_path

* refine resource_path comments

* rename job resource path to code search path

* Add instruction about starting a cross-language cluster

* fix ClassLoaderTest.java

* add code-search-path to RunManager

* refine comments for code-search-path

* rename resourcePath to codeSearchPath

* Update doc

* fix

* rename resourcePath to codeSearchPath

* update doc

* filter out empty path

* fix comments

* fix comments

* fix tests

* revert pom

* lint

* fix doc

* update doc

* Apply suggestions from code review

* lint

Co-authored-by: Kai Yang <kfstorm@outlook.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>
This commit is contained in:
chaokunyang
2020-09-09 00:46:32 +08:00
committed by GitHub
parent 3645a05644
commit bbfbc98a41
22 changed files with 226 additions and 102 deletions
@@ -22,10 +22,10 @@ ray {
// If worker.mode is DRIVER, specify the job id.
// If not provided, a random id will be used.
id: ""
// If this config is set, worker will use different paths to load resources when
// executing tasks from different jobs. E.g. if it's set to '/tm/job_resources',
// the path for job 123 will be '/tmp/job_resources/123'.
resource-path: ""
// A list of directories or jar files separated by colon that specify the
// search path for user code. This will be used as `CLASSPATH` in Java,
// and `PYTHONPATH` in Python.
code-search-path: ""
/// The number of java worker per worker process.
num-java-workers-per-process: 1
/// The jvm options for java workers of the job.