Commit Graph

6812 Commits

Author SHA1 Message Date
Siyuan (Ryans) Zhuang 1c992661a8 Add scripts symlink back (#9219) (#9475)
(cherry picked from commit 77933c922d)

Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-07-14 12:31:49 -07:00
SangBin Cho 539c51a003 [Core] Support GCS server port assignment. (#8962) 2020-07-14 11:49:56 -05:00
SangBin Cho f6eb47fc1f [Stats] metrics agent exporter (#9361) 2020-07-14 11:49:16 -05:00
Siyuan (Ryans) Zhuang 5b192842b5 Fix ObjectRef and ActorHandle serialization (#9462) 2020-07-14 09:42:32 -07:00
Siyuan (Ryans) Zhuang d57ff5e2af Remove legacy C++ code (#9459) 2020-07-14 00:57:42 -07:00
krfricke deba082cb4 [tune] PyTorch CIFAR10 example (#9338)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-07-13 23:16:05 -07:00
kisuke95 276fe109c5 change error code name of boost timer (#9417) 2020-07-14 11:50:58 +08:00
fangfengbin 3c90f960fb Fix gcs_pubsub_test bug(#9438)
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-14 11:34:50 +08:00
Sven Mika 617eb8f279 [RLlib] Issue 9402 MARWIL producing nan rewards. (#9429) 2020-07-14 05:07:16 +02:00
Sven Mika 03ab86567f [RLlib] Layout of Trajectory View API (new class: Trajectory; not used yet). (#9269) 2020-07-14 04:27:49 +02:00
Max Fitton 222635b63f Machine View Sorting / Grouping (#9214)
* Convert NodeInfo.tsx to a functional component

* Update NodeRowGroup to be a functional component

* lint

* Convert TotalRow to functional component.

* lint

* move node info over to using the sortable table head component. spacing is still a little wonky.

* Factor a NoewWorkerRow class out of NodeRowGroup that will be usable when grouping / ungrouping

* Compilation checkpoint, I factored the worker filtering logic out of node info into the reducer

* Add sort accessors for CPU

* Add sort accessors for Disk

* Add sort accessors for RAM

* add a table sort util for function based accessors (rather than flat attribute-based accessor)

* wip refactor node info features

* wip

* Rendering Checkpoint. I've refactored the features and how they are called to add sorting support. Also reworks the way error counts and log counts are passed to the front-end to remove some ugly logic

* wip

* wip

* wip

* Finish adding sorting and grouping of machine view

* lint

* fix bug in filtration of logs and errors by worker from recent refactor.

* Add export of Cluster Disk feature

* fix some merge issues

Co-authored-by: Max Fitton <max@semprehealth.com>
2020-07-13 20:45:17 -05:00
SangBin Cho 22b2e51152 Fix test-multi-node (#9453) 2020-07-13 20:44:27 -05:00
Richard Liaw a567f7977c [tune] Put examples under proper version control (#9427)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-07-13 18:01:10 -07:00
Richard Liaw 7abf7a0109 [docs] Render ActorPool documentation, etc (#9433) 2020-07-13 17:59:22 -07:00
Vasily Litvinov 6ad13e0da8 Add ability to specify SOCKS proxy for SSH connections (#8833) 2020-07-13 16:10:07 -07:00
Richard Liaw dfe3ebe4a2 [tune] sklearn comment out (#9454) 2020-07-13 16:06:44 -07:00
Siyuan (Ryans) Zhuang 4da97a7c99 [Core] Build raylet client as an independent component (#9434) 2020-07-13 16:00:32 -07:00
mehrdadn 3d65682e62 Bazel selects compiler flags based on compiler (#9313)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 15:31:46 -07:00
Henk Tillman c7714ca575 GCP authentication using oauth tokens (#9279) 2020-07-13 14:36:40 -07:00
Ian Rodney 0085cf75d0 Allow --lru-evict to be passed into ray start (#8959) 2020-07-13 14:09:39 -07:00
Amog Kamsetty 4454d05bcf [Tune] Trainable documentation fix (#9448) 2020-07-13 13:15:01 -07:00
Hao Chen e6225bdfa1 [GCS] Fix the bug about raylet receiving duplicate actor creation tasks (#9422) 2020-07-13 11:34:02 -07:00
mehrdadn 5291bf235b TRAVIS_PULL_REQUEST is false for non-PRs, not empty (#9439)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-13 14:52:40 +02:00
Nicolaus93 b5a6c57295 [tune] handling nan values (#9381) 2020-07-12 17:08:36 -07:00
Tanay Wakhare 15aa08a3d1 [RLLib] WindowStat bug fix (#9213)
* WindowStat error catching, which processes NaNs properly instead of erroring. This ought to resolve issue #7910.
https://github.com/ray-project/ray/issues/7910
2020-07-12 23:01:32 +02:00
Tanay Wakhare 3536d8e4b3 Masking error. With t*valid_mask, we get the error np.inf*0 = np.inf (#9407) 2020-07-12 22:59:35 +02:00
Siyuan (Ryans) Zhuang 381c242f6b [Core] Simplify Raylet Client (#9420) 2020-07-12 12:42:54 -07:00
Henk Tillman 8c985dc797 Update conda and ray wheel on GCP images (#9388) 2020-07-12 12:12:27 -07:00
Sven Mika fcdf410ae1 [RLlib] Tf2.x native. (#8752) 2020-07-11 22:06:35 +02:00
mehrdadn 5c853eaa6a Fix copy to workspace (#9400) 2020-07-11 14:27:56 +02:00
Ian Rodney 26fcda50e7 Pass run args to DockerCommandRunner (#9411) 2020-07-10 18:09:01 -07:00
Simon Mo d4a5d09dab [Serve] Merge router with HTTPProxy (#9225) 2020-07-10 13:52:48 -07:00
Siyuan (Ryans) Zhuang 1798deae94 [Core] Plasma RAII support (#9370) 2020-07-10 09:22:29 -07:00
SangBin Cho d8a0d76d02 Fix macos compliation bug (#9391)
* Fix.
2020-07-10 09:18:09 -07:00
Sven Mika 14160ca58c [RLlib] Issue #9366 (DQN w/o dueling produces invalid actions). (#9386) 2020-07-10 12:43:03 +02:00
Kai Yang c89b59cf48 Remove the RAY_CHECK in Worker::Port() (#9348) 2020-07-10 18:06:25 +08:00
Kai Yang a98cd0670e [Java] Improve JNI performance when submitting and executing tasks (#9032) 2020-07-10 17:51:07 +08:00
Hao Chen d49dadf891 Change Python's ObjectID to ObjectRef (#9353) 2020-07-10 17:49:04 +08:00
Tao Wang 6311e5a947 [HOTFIX] Fix compile direct_actor_transport_test on mac (#9403) 2020-07-10 17:19:34 +08:00
fangfengbin 35861f17a3 Fix gcs_table_storage testcase bug (#9393)
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com>
2020-07-10 16:16:28 +08:00
Hao Chen bed1be611e Fix flaky test_dynres.py (#9310) 2020-07-10 10:34:23 +08:00
mehrdadn dd2cc6eb48 Update hiredis and remove Windows patches (#9289)
Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 18:45:44 -07:00
Patrick Ames dc51b08c36 [autoscaler] Allow users to disable the cluster config cache (#8117)
* [autoscaler] Remove autoscaler config cache.

* [autoscaler] Add flag allowing users to explicitly disable the config cache.
2020-07-09 15:47:58 -07:00
Stefan Schneider 6db55ca8db [docs][rllib] Recommended workflow for training, saving, and testing (#9319) 2020-07-09 15:47:10 -07:00
Eric Liang 09b9b81ea4 [autoscaler] Move command runners into separate file and clean up interface. (#9340)
* cleanup

* wip

* fix imports

* fix lint
2020-07-09 15:40:56 -07:00
Zhuohan Li 8a76f4cbb5 [Core] put small objects in memory store (#8972)
* remove the put in memory store

* put small objects directly in memory store

* cast data type

* fix another place that uses Put to spill to plasma store

* fix multiple tests related to memory limits

* partially fix test_metrics

* remove not functioning codes

* fix core_worker_test

* refactor put to plasma codes

* add a flag for the new feature

* add flag to more places

* do a warmup round for the plasma store

* lint

* lint again

* fix warmup store

* Update _raylet.pyx

Co-authored-by: Eric Liang <ekhliang@gmail.com>
2020-07-09 15:39:40 -07:00
Alex Wu 34b85659d4 [Core] New scheduler fixes (#9186)
* .

* test_args passes

* .

* test_basic.py::test_many_fractional_resources causes ray to hang

* test_basic.py::test_many_fractional_resources causes ray to hang

* .

* .

* useful

* test_many_fractional_resources fails instead of hanging now :)

* Passes test_fractional_resources

* .

* .

* Some cleanup

* git is hard

* cleanup

* Fixed scheduling tests

* .

* .
2020-07-09 15:37:51 -07:00
Alisa f0a72ad985 [Core] Add placement group scheduler and some api in resource scheduler (#9039)
* Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

* change the bundle id and delete unit count in bundle

change vector<bundle_spec> to vector<shared_ptr<bundle_spec>>

Add placement group scheduler and some api of resource scheduler.
Merge fix cv hang in multithread variables race (#8984).

change the bundle id and delete unit count in bundle

remove CheckIfSchedulable()

add comments and fix the bug in resource

* fix placement group schedule

* add placement group scheduler and change some api in resource scheduler

* fix by the comments

* fix conflict

* fix lint

* fix lint

* fix bug in merge

* fix lint

Co-authored-by: Lingxuan Zuo <skyzlxuan@gmail.com>
2020-07-09 15:37:18 -07:00
Stephanie Wang 0389735d7a [core] Pass owner address from the workers to the raylet (#9299)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Add owner address to task args

* Make TaskArg a virtual class, remove multi args

* Set owner address for task args

* merge

* Fix tests

* Add ObjectRefs to task dependency manager, pass from task spec args

* tmp

* tmp

* Fix

* Add ownership info for task arguments

* Convert WaitForDirectActorCallArgs

* lint

* build

* update

* build

* java

* Move code

* build

* Revert "Fix Google log directory again (#9063)"

This reverts commit 275da2e400.

* Fix free

* fix tests

* Fix tests

* build

* build

* fix

* Change assertion to warning to fix java
2020-07-09 14:35:54 -07:00
mehrdadn 4687b807c4 Combine different severities into the same log files (#9230)
* Combine different severities into the same log files

Co-authored-by: Mehrdad <noreply@github.com>
2020-07-09 14:14:28 -07:00