Commit Graph

76 Commits

Author SHA1 Message Date
SangBin Cho 539c51a003 [Core] Support GCS server port assignment. (#8962) 2020-07-14 11:49:56 -05:00
SangBin Cho f6eb47fc1f [Stats] metrics agent exporter (#9361) 2020-07-14 11:49:16 -05:00
Ian Rodney 0085cf75d0 Allow --lru-evict to be passed into ray start (#8959) 2020-07-13 14:09:39 -07:00
Hao Chen d49dadf891 Change Python's ObjectID to ObjectRef (#9353) 2020-07-10 17:49:04 +08:00
Ian Rodney 9172f8c3a6 [core] Store Internal Config in GCS (#8921) 2020-07-08 11:22:08 -05:00
Xianyang Liu 0bfcc2e5ba [core] Better support multi-nic environments by respecting user-provided IP (#8512) 2020-06-25 14:03:12 -05:00
Alex Wu c152730e4a [Core] Log output from different jobs to different drivers. (#8885)
* .

* .

* Correct now

* No interactivity errors

* format

* Filtering

* lint

* .

* No more filtering

* Removed interactivity

* .

* .

* .

* .

* .

* .

* Redirection works

* formatting

* something broken?

* .

* Works

* formatting

* redirect output

* formatting

* formatting

* Fix file descriptor leakage

* format

* .

* .

* .

* .

* .

* Refactor

* .

* Only run on job switch

* .

* cleanup

* .

* ...

* Review

* .

* .

* .

* .

* whoops

* .

* Should fix bug

* .

* .

* addressed comments

* formatting

* formatting

* Fix typo

* .

* .

* .

* .

Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-33.us-west-2.compute.internal>
2020-06-23 18:45:32 -07:00
Max Fitton ad09aa985c Make Dashboard Port Configurable (#8999) 2020-06-19 16:26:22 -05:00
Zhilei Chen d8a9247448 Remove gcs_service_disabled ci jobs and code (#8854) 2020-06-19 11:32:27 +08:00
Siyuan (Ryans) Zhuang 4b31b383f3 [Core] Run Plasma Store as a Raylet thread (with a feature flag) (#8897)
* integrate plasma store as a thread (C++)

* integrate plasma store as a thread (Python)

* fix config issues

* remove plasma component fail tests

* without forcefully kill the plasma store thread
2020-06-11 22:54:08 -07:00
Siyuan (Ryans) Zhuang 2f690d1866 Simplify plasma store config (#8823)
* simplify plasma store config
2020-06-08 20:47:22 -07:00
fangfengbin 68718b33b4 GCS Server add SIGTERM signal handler (#8795) 2020-06-08 17:26:36 +08:00
Ian Rodney 7a2c9524d1 [Core] Randomize and 'Reserve' Port Generated for Node Manager (#8628) 2020-06-03 12:19:03 -07:00
Edward Oakes a76434ccde Add ability to specify worker and driver ports (#8071) 2020-05-20 15:31:13 -05:00
Edward Oakes 90ef585fd5 Revert "Add ability to specify worker and driver ports (#7833)" (#8069)
This reverts commit 9f751ff8c4.
2020-04-17 12:32:22 -05:00
Edward Oakes 9f751ff8c4 Add ability to specify worker and driver ports (#7833) 2020-04-16 13:49:25 -05:00
Clark Zinzow d4cae5f632 [Core] Added ability to specify different IP addresses for a core worker and its raylet. (#7985) 2020-04-16 10:32:24 -05:00
mehrdadn 1b0f6fd558 Check AF_UNIX path length (#7951) 2020-04-13 09:30:01 -07:00
mehrdadn 0b4e09da76 Log to terminal if glog is also doing so (#7868) 2020-04-10 18:41:21 -05:00
ijrsvt e03f687b84 Cleaning up remaining Local Mode Code (#7865) 2020-04-03 19:54:15 -05:00
ijrsvt 9bfc2c4b54 Moving Local Mode to C++ (#7670) 2020-04-01 15:50:57 -05:00
mehrdadn fc23f79f82 Windows process issues (#7739) 2020-03-29 12:48:32 -07:00
fangfengbin e196fcdbaf Add gcs_service_enabled function to avoid getting environment variable directly (#7742) 2020-03-26 22:02:53 +08:00
fangfengbin bf866de6fd Enable GCS Service by default (#7541) 2020-03-24 14:20:23 +08:00
mehrdadn b4030cdbbe File HANDLE/descriptor translation layer for Windows (#7657)
* Use TCP sockets on Windows with custom HANDLE <-> FD translation layer

* Get Plasma working on Windows

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-23 21:08:25 -07:00
mehrdadn a0700e2f86 Change /tmp to platform-specific temporary directory (#7529) 2020-03-16 18:10:14 -07:00
Stephanie Wang 53549314c5 [core] Option to fallback to LRU on OutOfMemory (#7410)
* Add a test for LRU fallback

* Update error message

* Upgrade arrow to master

* Integrate with arrow

* Revert "Bazel mirrors (#7385)"

This reverts commit 44aded5272.

* Don't LRU evict

* Revert "Revert "Bazel mirrors (#7385)""

This reverts commit b6359fea78d1bd3925452ca88ac71e0c9e5c7dd3.

* Add lru_evict flag

* fix internal config

* Fix

* upgrade arrow

* debug

* Set free period in config for lru_evict, override max retries to fix
test

* Fix test?

* fix test

* Revert "debug"

This reverts commit 98f01c63a267f38218f5047b1866e4c1c8280017.

* fix exception str

* Fix ref count test

* Shorten travis test?
2020-03-14 11:28:43 -07:00
ZhuSenlin b663bc6d67 Use gcs server to replace raylet monitor when RAY_GCS_SERVICE_ENABLED=true (#7166) 2020-03-12 22:13:56 +08:00
mehrdadn 3b9caa98ba Fix fate-sharing warning (#7545)
* Fix kernel_fate_sharing being None instead of False

* Remove fate-sharing warning

Co-authored-by: Mehrdad <noreply@github.com>
2020-03-11 08:27:54 -07:00
Edward Oakes 7b609ca211 Remove instances of 'raise Exception' (#7523) 2020-03-10 17:51:22 -07:00
Edward Oakes 4ab80eafb9 Deprecate use_pickle flag (#7474) 2020-03-09 16:03:56 -07:00
mehrdadn 4d42664b2a Use prctl(PR_SET_PDEATHSIG) on Linux instead of reaper (#7150) 2020-03-03 11:45:42 -06:00
fangfengbin e7d0ec9531 Enable GCS server when running python unit tests (#7101)
* Enable GCS server when running python unit tests

* restart ci

* restart ci

* fix code style

* restart ci

* restart ci

* restart ci

* restart ci

* restart ci

* Define RAY_GCS_SERVICE_ENABLED as a constant

* fix review comments

* fix code style

* fix code style

* fix code style

* fix code style

* fix review comments

* add gcs service python testcase

* fix TESTSUITE name bug
2020-02-24 09:48:40 +08:00
fangfengbin 694c0f2867 [Java] Enable GCS server when running java unit tests (#7041)
* enable gcs service when run java testcase

* fix ci bug

* fix windows compile bug

* fix ci bug

* restart ci job

* enable java testcase

* restart ci job

* restart ci job

* add debug log

* add debug log

* restart ci job

* add debug log

* restart ci

* add debug log

* fix java testcase bug

* restart ci job

* restart ci job

* restart ci job

* restart ci job

* restart ci job

* restart ci job

* restart ci job

* restart ci job
2020-02-10 09:39:14 +08:00
Sven 60d4d5e1aa Remove future imports (#6724)
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
Ujval Misra 5b40408678 [tune] Remove py2.7-specific code (#6665)
* Remove backwards compatability py2.7 code.

* Use exists_ok=True in ray

* nit

* nit

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-01-03 01:03:13 -08:00
Robert Nishihara 39a3459886 Remove (object) from class declarations. (#6658) 2020-01-02 17:42:13 -08:00
Robert Nishihara 480206eef8 Remove some Python 2 compatibility code. (#6624) 2019-12-31 17:14:58 -08:00
Robert Nishihara 8724e5ffd5 Start WebUI by default. (#6493) 2019-12-27 13:49:07 -08:00
Edward Oakes e4f9b3b7d9 Use process reaper for cleanup (#6253) 2019-11-26 22:00:08 -06:00
Edward Oakes e3b95dafeb Fix sigterm_handler (#6141) 2019-11-14 13:41:50 -08:00
Philipp Moritz f24d96ec4f Revert "Try to enable dashboard (again) (#6069)" (#6159)
This reverts commit 4044af8520.
2019-11-13 12:32:12 -08:00
Stephanie Wang 35d177f459 Use grpc for communication from worker to local raylet (task submission and direct actor args only) (#6118)
* Skeleton for SubmitTask proto

* Pass through node manager port, connect in raylet client

* Switch submit task to grpc

* Check port in use

* doc

* Remove default port, set port randomly from driver

* update

* Fix test

* Fix object manager test
2019-11-11 21:17:25 -08:00
Eric Liang 4044af8520 Try to enable dashboard (again) (#6069)
* Revert "Revert "Enable the Ray dashboard by default (#5976)" (#6068)"

This reverts commit 1a3e97cf23.

* fix tests that assume the dashboard isn't a job

* travis
2019-11-08 10:48:48 -08:00
Eric Liang 1a3e97cf23 Revert "Enable the Ray dashboard by default (#5976)" (#6068)
This reverts commit 6166ef3e09.
2019-11-01 17:08:37 -07:00
Eric Liang 6166ef3e09 Enable the Ray dashboard by default (#5976) 2019-11-01 12:19:01 -07:00
Mitchell Stern 235dec8aa3 [Dashboard] Remove token authentication from dashboard (#5888) 2019-10-21 12:48:48 -07:00
Philipp Moritz d23696de17 Introduce flag to use pickle for serialization (#5805) 2019-10-18 22:29:36 -07:00
Edward Oakes a8888c5ff4 [flaky test] Fix test_calling_start_ray_head (#5644) 2019-09-14 22:27:45 -07:00
Edward Oakes 07c4c6367a [core worker] Python core worker object interface (#5272) 2019-09-12 23:07:46 -07:00