wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-13 09:58:58 +08:00
Code Issues Packages Projects Releases Wiki Activity
4,968 Commits 36 Branches 42 Tags
56d934bc188d24e2b658d774a23e5e4709315b50
Commit Graph

412 Commits

Author SHA1 Message Date
Stephanie Wang 490cddc250 [core] Refactor distributed ref counting to remove owner task ID (#9049) 
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Update message

* lint

* Fix build
 2020-06-25 17:55:03 -07:00
Xianyang Liu 0bfcc2e5ba [core] Better support multi-nic environments by respecting user-provided IP (#8512) 2020-06-25 14:03:12 -05:00
Alex Wu 46962f5db1 [Core] Log monitor multidriver (#8953) 2020-06-25 11:05:53 -07:00
Eric Liang 4522038259 [iter] Add .transform() function for arbitrary generator transforms (#8978) 2020-06-25 11:04:14 -07:00
Eric Liang 536795ef79 [autoscaler] Initial support for multiple worker types (#9096) 
* wip

* fix

* update

* debug state

* fix

* update

* update

* fix test

* fix

* fix

* update

* fix

* types and docs

* update
 2020-06-24 23:08:30 -07:00
Eric Liang 0ff24ec8dc Add "ray status" debug tool for autoscaler. (#9091) 2020-06-24 18:22:03 -07:00
Alex Wu c152730e4a [Core] Log output from different jobs to different drivers. (#8885) 
* .

* .

* Correct now

* No interactivity errors

* format

* Filtering

* lint

* .

* No more filtering

* Removed interactivity

* .

* .

* .

* .

* .

* .

* Redirection works

* formatting

* something broken?

* .

* Works

* formatting

* redirect output

* formatting

* formatting

* Fix file descriptor leakage

* format

* .

* .

* .

* .

* .

* Refactor

* .

* Only run on job switch

* .

* cleanup

* .

* ...

* Review

* .

* .

* .

* .

* whoops

* .

* Should fix bug

* .

* .

* addressed comments

* formatting

* formatting

* Fix typo

* .

* .

* .

* .

Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-33.us-west-2.compute.internal>
 2020-06-23 18:45:32 -07:00
Alex Wu 40c15b1ba0 [ParallelIterator] Fix for_each concurrent test cases/bugs (#8964) 
* Everything works

* Update python/ray/util/iter.py

Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>

* .

* .

* removed print statements

Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
 2020-06-22 18:26:45 -07:00
mehrdadn 981f67bfb0 Fix more Windows issues (#9011) 
Co-authored-by: Mehrdad <noreply@github.com>
 2020-06-19 18:51:45 -07:00
Max Fitton ad09aa985c Make Dashboard Port Configurable (#8999) 2020-06-19 16:26:22 -05:00
Zhilei Chen d8a9247448 Remove gcs_service_disabled ci jobs and code (#8854) 2020-06-19 11:32:27 +08:00
Edward Oakes 8a99fd205e [dashboard] Pipe resource assignments to dashboard (#8998) 2020-06-18 11:14:59 -05:00
Alan Guo 8f9b465968 Extract out "fillout_defaults" from other config preparation actions (#8996) 2020-06-17 16:40:37 -07:00
mehrdadn 101c215125 Get more tests running on Windows (#6537) 
* Get rid of system() calls

* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)

https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true

* Don't check for socket max path length on Windows

* Don't check for socket existence on Windows

* Fix race condition in Windows fate-sharing

* Work around missing .exe extension for Redis tests

* Add more tests to GitHub Actions

Co-authored-by: Mehrdad <noreply@github.com>
 2020-06-12 21:32:10 -07:00
Siyuan (Ryans) Zhuang 4b31b383f3 [Core] Run Plasma Store as a Raylet thread (with a feature flag) (#8897) 
* integrate plasma store as a thread (C++)

* integrate plasma store as a thread (Python)

* fix config issues

* remove plasma component fail tests

* without forcefully kill the plasma store thread
 2020-06-11 22:54:08 -07:00
Stephanie Wang 05010caed2 [core] Fix race condition for object reconstruction (#8791) 
* Fix

* doc

* Unit test

* Update src/ray/core_worker/task_manager.h

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

* Update src/ray/core_worker/task_manager.h

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

* Update src/ray/core_worker/task_manager.h

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>

* lint

Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
 2020-06-10 19:49:12 -07:00
fangfengbin a5bebd4408 Fix create actor rpc reconnect bug (#8855) 2020-06-10 10:53:53 +08:00
SangBin Cho 731ed8d232 [Core] Fix a detached actor bug fix when GCS actor management is off. (#8843) 2020-06-09 15:46:17 -07:00
Kai Yang db5cc5c8da fix test_global_state_api due to the temporary object (#8800) 
* fix test_global_state_api due to the temporary object

* update

* Update python/ray/tests/test_advanced_3.py

Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>

Co-authored-by: Robert Nishihara <robertnishihara@gmail.com>
 2020-06-09 11:42:40 +08:00
Simon Mo 5e2bd6ecb9 Fix asyncio re-entry error message (#8842) 2020-06-08 17:43:01 -07:00
SangBin Cho 3388864768 [Core] Clean up detached actors (#8759) 2020-06-08 11:22:01 -05:00
Stephanie Wang b160b83d3e [core] Queue subscription/unsubscription commands in the GCS (#8756) 
* Only remove callback index if in map

* test

* Queue subscription commands

* lint

* Check status

* update

* update

* update

* Disable GCS restart tests

* lint
 2020-06-05 19:49:19 -07:00
SangBin Cho e372c06257 Hotfix dashboard broken tests. (#8757) 2020-06-04 09:44:00 -07:00
fangfengbin 84a8f2ccb5 Support reloading storage data when gcs server restarts (#8650) 2020-06-04 14:53:20 +08:00
Eric Liang a24d117c68 [autoscaler] Refactor code in preparation for multi instance type support (#8632) 
* wip refactor

* add util

* wip

* fix

* fix

* remove

* remove extraneous string type for sg
 2020-06-03 12:53:55 -07:00
Max Fitton b9f0f7ae5b Dashboard minor refactor and first unit tests (#8705) 2020-06-03 11:04:55 -05:00
SangBin Cho 7c43991100 [GCS] Monitor.py bug fix (#8725) 
* comment.

* Fix bugs.

* Used pubsub message instead.

* Added a ray.actors test
 2020-06-02 16:06:36 -07:00
Stephanie Wang aa06c3b15a Eager eviction even when object pinning is disabled (#8561) 
* Eager eviction even when object pinning is disabled, add regression test

* Make test more robust

* lint
 2020-06-02 11:48:03 -07:00
fangfengbin 016337d4eb Heartbeat table uses gcs pub-sub instead of redis accessor (#8655) 2020-05-30 23:17:25 +08:00
SangBin Cho 1115231e7c [Test] Fix test_reconstruction OOM error (#8636) 2020-05-29 18:56:19 -05:00
Sven Mika d483ed28ba [RLlib] Fix broken tune tests in master due to framework=auto errors. (#8672) 2020-05-29 11:55:47 +02:00
Patrick Ames 76450c8d47 [autoscaler] Honor separate head and worker node subnet IDs (#8374) 2020-05-28 18:16:46 -07:00
Ian Rodney 99cc2e28b3 [Core] Fix test_cancel (#8644) 2020-05-28 09:40:37 -07:00
Lingxuan Zuo e594524ed3 [GCS] global state query node info table from GCS. (#8498) 2020-05-28 16:39:13 +08:00
fyrestone b0bb0584fb Fix fix_test_actor_method_metadata_cache (#8633) 2020-05-27 15:49:40 +08:00
mehrdadn 79a4eac48c Make more tests run on Windows (#8553) 2020-05-26 18:43:34 -05:00
Amog Kamsetty ae2e1f0883 [Parallel Iterators] Batching + Pipelining optimizations (#7931) 
* batching + get_shard pipelining

* duplicate fix

* formatting

* adding performance benchmark

* minor changes

* turn batching off by default
 2020-05-26 00:37:57 -07:00
fangfengbin 765d470c40 Add gcs object manager (#8298) 2020-05-25 17:21:35 +08:00
Edward Oakes 860eb6f13a Update named actor API (#8559) 2020-05-24 20:08:03 -05:00
SangBin Cho aa1cbe8abc [Dashboard] Ray memory dashboard backend (#8461) 2020-05-21 12:22:28 -07:00
Eric Liang 9a83908c46 [rllib] Deprecate policy optimizers (#8345) 2020-05-21 10:16:18 -07:00
Edward Oakes a76434ccde Add ability to specify worker and driver ports (#8071) 2020-05-20 15:31:13 -05:00
mehrdadn ebf060d484 Make more tests run on Windows (#8446) 
* Remove worker Wait() call due to SIGCHLD being ignored

* Port _pid_alive to Windows

* Show PID as well as TID in glog

* Update TensorFlow version for Python 3.8 on Windows

* Handle missing Pillow on Windows

* Work around dm-tree PermissionError on Windows

* Fix some lint errors on Windows with Python 3.8

* Simplify torch requirements

* Quiet git clean

* Handle finalizer issues

* Exit with the signal number

* Get rid of wget

* Fix some Windows compatibility issues with tests

Co-authored-by: Mehrdad <noreply@github.com>
 2020-05-20 12:25:04 -07:00
Ian Rodney 1163ddbe45 Remove timeouts in test_cancel (#8272) 2020-05-19 12:35:16 -05:00
Max Fitton 13231ba63b Rename redis-port to port and add default (#8406) 2020-05-18 13:25:34 -05:00
Joseph Lucas 42c9fa19d1 [autoscaler] Ray Up url-arg (#8279) 2020-05-17 12:18:00 -07:00
Edward Oakes 16f48078d9 Remove use of ObjectID transport flag (#7699) 2020-05-17 11:29:49 -05:00
Stephanie Wang bd169749e0 Option to retry failed actor tasks (#8330) 
* Python

* Consolidate state in the direct actor transport, set the caller starts at

* todo

* Remove unused

* Update and unit tests

* Doc

* Remove unused

* doc

* Remove debug

* Update src/ray/core_worker/transport/direct_actor_transport.h

Co-authored-by: Eric Liang <ekhliang@gmail.com>

* Update src/ray/core_worker/transport/direct_actor_transport.cc

Co-authored-by: Eric Liang <ekhliang@gmail.com>

* lint and fix build

* Update

* Fix build

* Fix tests

* Unit test for max_task_retries=0

* Fix java?

* Fix bad test

* Cross language fix

* fix java

Co-authored-by: Eric Liang <ekhliang@gmail.com>
 2020-05-15 20:15:15 -07:00
Max Fitton 00325eb2b2 Rename max_reconstructions to max_restarts and use -1 for infinite (#8274) 
Co-authored-by: Edward Oakes <ed.nmi.oakes@gmail.com>
 2020-05-14 10:30:29 -05:00
Edward Oakes b84fe56bed Split test_basic to avoid timeouts in CI (#8405) 2020-05-12 10:18:21 -05:00