Commit Graph

248 Commits

Author SHA1 Message Date
Eric Liang 609c1b8acd Start moving ray internal files to _private module (#10994) 2020-09-24 22:46:35 -07:00
Keqiu Hu 46a560e876 [cli][ray] ray start should error by default if there's already an instance running (#10826) 2020-09-24 10:29:59 -05:00
SongGuyang f9b040db52 add log-dir to new dashboard (#10885) 2020-09-24 13:40:37 +08:00
SangBin Cho 390107b6cb [Core] Allow to pass node ip address to gcs server. (#10946)
* Allow to pass node ip address to gcs server.

* Fix.

* Addressed code review.

* Fixed an error.

* Addressed code review.
2020-09-23 01:52:26 -07:00
SongGuyang 5cbc411e38 [cpp worker] support cluster mode (#9977) 2020-09-18 11:08:18 +08:00
SangBin Cho fe4c6ab778 [Core] Remove unused credis related code. (#10849)
* Done.

* Lint.
2020-09-16 23:34:54 -07:00
Hao Chen 1a5cfe0b79 [Java] rename config ray.redis.address to ray.address (#10772) 2020-09-15 11:13:19 +08:00
Barak Michener 9f9b53e624 Remove superfluous execution of java (#10750) 2020-09-11 19:03:38 -07:00
Eric Liang cbc52a2a42 Remove colorful from ray core (#10723) 2020-09-10 18:41:59 -07:00
Kai Yang afa0216280 Remove the '--include-java' option (#10594) 2020-09-09 17:01:17 +08:00
Alex Wu d9c68fca5c [Core] Logging improvements (#10625)
* other stuff
:

* lint

* .

* .

* lint

* comment

* lint

* .
2020-09-08 20:58:05 -07:00
chaokunyang bbfbc98a41 [Core] Allow users to specify the classpath and import path (#10560)
* move job resource path to job config

* job resource path support list

* job resource path support for python

* fix job_resource_path support

* fix worker command

* fix job config

* use jar file instead of parent path

* fix job resource path

* add test to test.sh

* lint

* Update java/runtime/src/main/resources/ray.default.conf

Co-authored-by: Kai Yang <kfstorm@outlook.com>

* fix testGetFunctionFromLocalResource

* lint

* fix rebase

* add jars in resource path to classloader

* add job_resource_path to worker

* add ray stop

* rename job_resource_path to resource_path

* fix resource_path

* refine resource_path comments

* rename job resource path to code search path

* Add instruction about starting a cross-language cluster

* fix ClassLoaderTest.java

* add code-search-path to RunManager

* refine comments for code-search-path

* rename resourcePath to codeSearchPath

* Update doc

* fix

* rename resourcePath to codeSearchPath

* update doc

* filter out empty path

* fix comments

* fix comments

* fix tests

* revert pom

* lint

* fix doc

* update doc

* Apply suggestions from code review

* lint

Co-authored-by: Kai Yang <kfstorm@outlook.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>
2020-09-09 00:46:32 +08:00
fyrestone e9b046306a [Dashboard] Dashboard basic modules (#10303)
* Improve reporter module

* Add test_node_physical_stats to test_reporter.py

* Add test_class_method_route_table to test_dashboard.py

* Add stats_collector module for dashboard

* Subscribe actor table data

* Add log module for dashboard

* Only enable test module in some test cases

* CI run all dashboard tests

* Reduce test timeout to 10s

* Use fstring

* Remove unused code

* Remove blank line

* Fix dashboard tests

* Fix asyncio.create_task not available in py36; Fix lint

* Add format_web_url to ray.test_utils

* Update dashboard/modules/reporter/reporter_head.py

Co-authored-by: Max Fitton <mfitton@berkeley.edu>

* Add DictChangeItem type for Dict change

* Refine logger.exception

* Refine GET /api/launch_profiling

* Remove disable_test_module fixture

* Fix test_basic may fail

Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: Max Fitton <mfitton@berkeley.edu>
2020-08-29 23:09:34 -07:00
Eric Liang 519354a39a [api] Initial API deprecations for Ray 1.0 (#10325) 2020-08-28 15:03:50 -07:00
SongGuyang cb70864c04 [cpp worker] support cluster mode and object Put/Get works (#9682) 2020-08-28 13:53:36 +08:00
fyrestone 05c103af94 [Dashboard] Start the new dashboard (#10131)
* Use new dashboard if environment var RAY_USE_NEW_DASHBOARD exists; new dashboard startup

* Make fake client/build/static directory for dashboard

* Add test_dashboard.py for new dashboard

* Travis CI enable new dashboard test

* Update new dashboard

* Agent manager service

* Add agent manager

* Register agent to agent manager

* Add a new line to the end of agent_manager.cc

* Fix merge; Fix lint

* Update dashboard/agent.py

Co-authored-by: SangBin Cho <rkooo567@gmail.com>

* Update dashboard/head.py

Co-authored-by: SangBin Cho <rkooo567@gmail.com>

* Fix bug

* Add tests for dashboard

* Fix

* Remove const from Process::Kill() & Fix bugs

* Revert error check of execute_after

* Raise exception from DashboardAgent.run

* Add more tests.

* Fix compile on Linux

* Use dict comprehension instead of dict(generator)

* Fix lint

* Fix windows compile

* Fix lint

* Test Windows CI

* Revert "Test Windows CI"

This reverts commit 945e01051ec95cff5fcc1c0bc37045b46e7ad9a6.

* Fix ParseWindowsCommandLine bug

* Update src/ray/util/util.cc

Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>

Co-authored-by: 刘宝 <po.lb@antfin.com>
Co-authored-by: SangBin Cho <rkooo567@gmail.com>
Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>
2020-08-24 13:24:23 -07:00
SangBin Cho 92664249e8 Partially Use f string (#10218)
* flynt. trial 1.

* Trial 1.

* Addressed code review.
2020-08-20 18:21:16 -07:00
Siyuan (Ryans) Zhuang 17ca1d8ff4 [Core] Object spilling prototype (#9818) 2020-08-14 15:39:10 -07:00
Robert Nishihara 36e626e95d Revert "[Dashboard] Start the new dashboard (#9860)" (#10116)
This reverts commit 739933e5b8.
2020-08-14 14:06:57 -07:00
Simon Mo 01f38bc5d1 CoreWorker correctly push metrics to agent (#10031) 2020-08-13 16:44:53 -07:00
fyrestone 739933e5b8 [Dashboard] Start the new dashboard (#9860) 2020-08-13 11:01:46 +08:00
Maksim Smolin 40b8e35d61 [cli] New logging for the rest of the ray commands (#9984)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-08-11 09:58:23 -07:00
Kai Yang 37821f0b4c Support unlimited JVM options (#9910) 2020-08-10 16:08:33 +08:00
SangBin Cho ec2f1a225e [Stats] Metrics Export User Interface Part 1 (#9913)
* Metrics export port expose done.

* Support exposing metrics port + metrics agent service discovery through ray.nodes()

* Formatting.

* Added a doc.

* Linting.

* Change the location of metrics agent port.

* Addressed code review.

* Addressed code review.
2020-08-06 16:16:29 -07:00
Kai Yang 27cd323ce1 [Core] Multi-tenancy: Job isolation & implement per job config (except for env variables) (#9500) 2020-08-04 15:51:29 +08:00
SangBin Cho d49b19c24c [Stats] Improve Stats::Init & Add it to GCS server (#9563) 2020-07-25 10:42:08 -07:00
SangBin Cho 539c51a003 [Core] Support GCS server port assignment. (#8962) 2020-07-14 11:49:56 -05:00
SangBin Cho f6eb47fc1f [Stats] metrics agent exporter (#9361) 2020-07-14 11:49:16 -05:00
Kai Yang fa1eadd61f fix java worker log dir (#9352) 2020-07-09 10:11:03 +08:00
Ian Rodney 9172f8c3a6 [core] Store Internal Config in GCS (#8921) 2020-07-08 11:22:08 -05:00
mehrdadn 7135cb2aec Fix .exe file extensions (#9197)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-02 15:29:34 -05:00
ChenZhilei c11855728a Remove raylet monitor after use GCS service (#9179) 2020-07-01 20:01:52 +08:00
Xianyang Liu 0bfcc2e5ba [core] Better support multi-nic environments by respecting user-provided IP (#8512) 2020-06-25 14:03:12 -05:00
Max Fitton ad09aa985c Make Dashboard Port Configurable (#8999) 2020-06-19 16:26:22 -05:00
henktillman 508149b3c3 Remove redundant logger warning (#8954) 2020-06-15 21:14:58 -07:00
mehrdadn 101c215125 Get more tests running on Windows (#6537)
* Get rid of system() calls

* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)

https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true

* Don't check for socket max path length on Windows

* Don't check for socket existence on Windows

* Fix race condition in Windows fate-sharing

* Work around missing .exe extension for Redis tests

* Add more tests to GitHub Actions

Co-authored-by: Mehrdad <noreply@github.com>
2020-06-12 21:32:10 -07:00
Siyuan (Ryans) Zhuang 4b31b383f3 [Core] Run Plasma Store as a Raylet thread (with a feature flag) (#8897)
* integrate plasma store as a thread (C++)

* integrate plasma store as a thread (Python)

* fix config issues

* remove plasma component fail tests

* without forcefully kill the plasma store thread
2020-06-11 22:54:08 -07:00
Siyuan (Ryans) Zhuang 2f690d1866 Simplify plasma store config (#8823)
* simplify plasma store config
2020-06-08 20:47:22 -07:00
Ian Rodney 7a2c9524d1 [Core] Randomize and 'Reserve' Port Generated for Node Manager (#8628) 2020-06-03 12:19:03 -07:00
Lingxuan Zuo e594524ed3 [GCS] global state query node info table from GCS. (#8498) 2020-05-28 16:39:13 +08:00
Edward Oakes a76434ccde Add ability to specify worker and driver ports (#8071) 2020-05-20 15:31:13 -05:00
Max Fitton 13231ba63b Rename redis-port to port and add default (#8406) 2020-05-18 13:25:34 -05:00
Robert Nishihara 2cff471d2c Don't print Redis connection warning in ray.init(). (#8475) 2020-05-18 11:19:13 -07:00
Edward Oakes 90ef585fd5 Revert "Add ability to specify worker and driver ports (#7833)" (#8069)
This reverts commit 9f751ff8c4.
2020-04-17 12:32:22 -05:00
Edward Oakes 9f751ff8c4 Add ability to specify worker and driver ports (#7833) 2020-04-16 13:49:25 -05:00
Clark Zinzow d4cae5f632 [Core] Added ability to specify different IP addresses for a core worker and its raylet. (#7985) 2020-04-16 10:32:24 -05:00
Qing Wang 98bfcd53bc [Java] Rename group id and package name. (#7864)
* Initial

* Change streaming's

* Fix

* Fix

* Fix org_ray

* Fix cpp file name

* Fix streaming

* Fix

* Fix

* Fix testlistening

* Fix missing sth in python

* Fix

* Fix

* Fix SPI

* Fix

* Fix complation

* Fix

* Fix CI

* Fix checkstyle

Fix checkstyle

* Fix streaming tests

* Fix streaming CI

* Fix streaming checkstyle.

* Fix build

* Fix bazel dep

* Fix

* Fix ray checkstyle

* Fix streaming checkstyle

* Fix bazel checkstyle
2020-04-12 17:59:34 +08:00
mehrdadn 0b4e09da76 Log to terminal if glog is also doing so (#7868) 2020-04-10 18:41:21 -05:00
SangBin Cho c23e56ce9a Metrics Export Service (#7809) 2020-03-30 23:28:32 -07:00
mehrdadn 8958728139 Windows bug fixes (#7740) 2020-03-30 20:39:23 -05:00