Commit Graph

204 Commits

Author SHA1 Message Date
SangBin Cho cb9fa90203 [Object Spilling] Add consumed bytes to detect thrashing. (#13853) 2021-02-03 14:16:26 -08:00
Eric Liang 26beb3b67b Revert "Revert "Enable Ray client server by default (#13350)" (#13429)" (#13442)
* Revert "Revert "Enable Ray client server by default (#13350)" (#13429)"

This reverts commit 560299972c.

* fix job id collision with ray client server
2021-02-02 00:17:29 -08:00
Simon Mo 8b8d6b984b [Buildkite] Add all Python tests (#13566) 2021-01-25 16:05:59 -08:00
Ameer Haj Ali 1fbc3ddfac Add ability to not start Monitor when calling ray start (#13505) 2021-01-18 18:31:53 -08:00
Micah Yong c89ebdd94a [Core][CLI] ray status and ray memory no longer starts a new job (#13391)
* Access memory info in ray memory via GlobalStateAccessor rather than calling ray.init()

* Modify ray status cli so that it doesn't start a new job via ray.init()

* Remove local test file

* Access memory info in ray memory via GlobalStateAccessor rather than calling ray.init()

* Modify ray status cli so that it doesn't start a new job via ray.init()

* Remove local test file

* Make status and error args required in commands.py#debug.status

* Remove unnecessary imports

* Access memory info in ray memory via GlobalStateAccessor rather than calling ray.init()

* Modify ray status cli so that it doesn't start a new job via ray.init()

* Remove local test file

* Access memory info in ray memory via GlobalStateAccessor rather than calling ray.init()

* Modify ray status cli so that it doesn't start a new job via ray.init()

* Remove local test file

* Make status and error args required in commands.py#debug.status

* Remove unnecessary imports

* Job 38482.1 should now pass

* Resolve merge conflict
2021-01-14 10:12:16 -08:00
Amog Kamsetty 560299972c Revert "Enable Ray client server by default (#13350)" (#13429)
This reverts commit 912d0cbbf9.
2021-01-13 21:28:54 -08:00
Eric Liang 912d0cbbf9 Enable Ray client server by default (#13350)
* update

* fix

* fix test

* update
2021-01-12 21:31:01 -08:00
Philipp Moritz 9872fc1801 Start ray client server with 'ray start' (#13217) 2021-01-06 21:04:14 -08:00
Philipp Moritz bf6577c8f4 Switch debugger to sockets and support unicode (#13004) 2020-12-20 12:10:28 -08:00
Eric Squires 22c1968d62 Runing -> Running (#12826) 2020-12-13 22:23:48 -08:00
Eric Squires 9f70293700 Remove debug extras from setup.py (#12751) 2020-12-10 16:23:11 -06:00
Kai Yang e3b5deb741 [Multi-tenancy] Delete flag enable_multi_tenancy and remove old code path (#10573) 2020-12-10 19:01:40 +08:00
Philipp Moritz 73a1a232b9 Ray debugger stepping between tasks (#12075) 2020-12-06 21:50:18 -08:00
Kai Yang 21fcee28f9 [Java] Simplify Ray.init() by invoking ray start internally (#10762) 2020-12-04 14:33:45 +08:00
Tao Wang e1075c0a82 [GCS]Fill resource fields when re-report heartbeat after gcs restarted (#12097) 2020-11-25 11:07:02 +08:00
Eric Liang e72abcd0aa Enable even more new scheduler tests (#12096) 2020-11-19 16:47:18 -08:00
Ian Rodney a74f1885db Revert "[CLI] Fix ray commands when RAY_ADDRESS used (#11989)" (#12135)
* Revert "[CLI] Fix ray commands when RAY_ADDRESS used (#11989)"

This reverts commit d23d326560.

* only check environment for CLI commands

* use new fns

* fixing docs

* rename and return "auto"

* Update python/ray/_private/services.py

Co-authored-by: Eric Liang <ekhliang@gmail.com>

* Update services.py

* Update services.py

Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-11-18 22:41:10 -08:00
SangBin Cho f56d7c1a76 [Logging] Remove per worker job log file / support worker log rotation (#11927)
* In progress.

* MVP done.

* In Progress.

* Remove unnecessay code.

* Fix some issues.

* Fix test failures.

* Addressed code review + fix object spilling test failure.
2020-11-16 11:29:43 -08:00
Philipp Moritz 28e7439cf0 [doc] Add documentation for Ray debugger (#11815)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-11-05 16:25:27 -08:00
Philipp Moritz 39ce0eadbe Ray PDB support (#11739) 2020-11-03 09:49:23 -08:00
Maksim Smolin 0a6d24a727 [cli] Remove the deprecated old_style logging calls (#10776)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-11-02 23:40:18 -08:00
Eric Liang 083737c63c Deprecate rsync to all nodes (#11563) 2020-10-22 13:45:42 -07:00
Edward Oakes 5d7f271e7d Add --worker-port-list option to ray start (#11481) 2020-10-21 14:46:45 -05:00
Maksim Smolin 94494c43d6 [cli] Updates (#10777) 2020-09-29 16:55:07 -07:00
Eric Liang 609c1b8acd Start moving ray internal files to _private module (#10994) 2020-09-24 22:46:35 -07:00
Keqiu Hu 46a560e876 [cli][ray] ray start should error by default if there's already an instance running (#10826) 2020-09-24 10:29:59 -05:00
Ian Rodney 4c3f09094a [docs] redis-port -> port (#10937) 2020-09-23 17:04:13 -07:00
Eric Liang 6a227ae501 [autoscaler] Split autoscaler interface public private (#10898) 2020-09-18 18:16:23 -07:00
Alex Wu 9a07c7b963 [1.0] Remove args from ray start (#10659) 2020-09-18 16:41:23 -07:00
Keqiu Hu 8a77cf925a [cli][ray] update ray cli message (#10823) 2020-09-17 09:26:55 -07:00
Siyuan (Ryans) Zhuang fb0ae148e6 Fix typo in ray start output (#10667) 2020-09-10 15:56:23 -07:00
Richard Liaw 7aa55ca82c [autoscaler] make commands very explicit on logs (#10713) 2020-09-10 10:47:46 -07:00
Alex Wu cd5b99e5e0 [hotfix] redis_password -> _redis_password (#10672) 2020-09-09 09:40:49 -07:00
Kai Yang afa0216280 Remove the '--include-java' option (#10594) 2020-09-09 17:01:17 +08:00
chaokunyang bbfbc98a41 [Core] Allow users to specify the classpath and import path (#10560)
* move job resource path to job config

* job resource path support list

* job resource path support for python

* fix job_resource_path support

* fix worker command

* fix job config

* use jar file instead of parent path

* fix job resource path

* add test to test.sh

* lint

* Update java/runtime/src/main/resources/ray.default.conf

Co-authored-by: Kai Yang <kfstorm@outlook.com>

* fix testGetFunctionFromLocalResource

* lint

* fix rebase

* add jars in resource path to classloader

* add job_resource_path to worker

* add ray stop

* rename job_resource_path to resource_path

* fix resource_path

* refine resource_path comments

* rename job resource path to code search path

* Add instruction about starting a cross-language cluster

* fix ClassLoaderTest.java

* add code-search-path to RunManager

* refine comments for code-search-path

* rename resourcePath to codeSearchPath

* Update doc

* fix

* rename resourcePath to codeSearchPath

* update doc

* filter out empty path

* fix comments

* fix comments

* fix tests

* revert pom

* lint

* fix doc

* update doc

* Apply suggestions from code review

* lint

Co-authored-by: Kai Yang <kfstorm@outlook.com>
Co-authored-by: Hao Chen <chenh1024@gmail.com>
2020-09-09 00:46:32 +08:00
Maksim Smolin f38dba09b2 [cli] Support non-interactive logging (#10203)
* add non-interactive logging

* logger

* add formatter support, add flag

* checkpoint

* format

* fix-interactive

* fixup

* fomrmat

* fix

* revert

* fix

* fix

* fix

* fix

* fx

* fix-test

* fix-commands

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-05 15:41:35 -07:00
Maksim Smolin 02efa2549d [cli] Fix colors on light themes (#10577) 2020-09-04 10:45:27 -07:00
chaokunyang ea95e6f7cc [Java] lint java code (#10494) 2020-09-03 10:39:14 +08:00
Richard Liaw 8c753818ab update-scripts (#10425) 2020-08-29 23:16:25 -07:00
Eric Liang 519354a39a [api] Initial API deprecations for Ray 1.0 (#10325) 2020-08-28 15:03:50 -07:00
Stephanie Wang f75dfd60a3 [api] API deprecations and cleanups for 1.0 (internal_config and Checkpointable actor) (#10333)
* remove

* internal config updates, remove Checkpointable

* Lower object timeout default

* remove json

* Fix flaky test

* Fix unit test
2020-08-27 10:19:53 -07:00
Maksim Smolin 245c0a9e43 [cli] Tests (#10057)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-08-22 13:29:10 -07:00
Richard Liaw f87669372d [cli] enable log-new-style by default (#10213) 2020-08-21 15:21:43 -07:00
SangBin Cho 92664249e8 Partially Use f string (#10218)
* flynt. trial 1.

* Trial 1.

* Addressed code review.
2020-08-20 18:21:16 -07:00
Max Fitton 8d06e30a06 [Dashboard] Fix Ray Dashboard command error messages (#10050) 2020-08-18 13:30:51 -05:00
Richard Liaw 7a56c3b71a [cli] create_or_update_cluster fix (#10085) 2020-08-13 00:54:45 -07:00
Richard Liaw 5560272556 [cli] install nightly wheels via ray install-nightly (#10054) 2020-08-11 20:08:22 -07:00
Maksim Smolin 40b8e35d61 [cli] New logging for the rest of the ray commands (#9984)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-08-11 09:58:23 -07:00
Maksim Smolin 0392bb7a72 [Autoscaler] Command Output Improvement (#9699)
* cross-platform prototype

* checkpoint

* Address PR comments

* format

* prepare to push

* format

* PR comments

* fixes

* fixtest

* Revert "fixtest"

This reverts commit d6f54353e1b891c784417bb8d0304c18cc5bcdd8.

* return-result

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-08-10 09:49:24 -07:00
SangBin Cho 39088ab6f2 [Stats] Metrics Export User Interface Part 2 (Prometheus Service Discovery) (#9970)
* In progress.

* In Progress.

* Finish the working version.

* Write a documentation.

* Addressed code review.

* Fix lint error.

* Lint.

* Addressed code review. Make test less flaky.

* Use a random port for ray start.

* Modify doc.

* Make write atomic.
2020-08-07 21:59:24 -07:00