Commit Graph
7159 Commits
Author SHA1 Message Date
architkulkarniandGitHub e89bbcbd44 [Serve] Revert "Revert "[Serve] Fix ServeHandle serialization"" and disable failing Windows test (#13771) 2021-02-04 14:50:01 -08:00
Edward OakesandGitHub 7af0c999f3 [serve] Built-in support for imported backends (#13867) 2021-02-04 15:09:12 -06:00
Dmitri GekhtmanandGitHub db59736b1a [autoscaler][kubernetes] Add ability to not copy cluster config to head node when calling create_or_update_head_node. (#13720)
* Add option to skip bootstrapping head node autoscaling config

* don't close remote config before copying

* Type

* Type hints etc.

* test

* Test CR to config conversion

* comment
2021-02-04 10:30:03 -08:00
Kai FrickeandGitHub 1e113d2e6e [tune/xgboost] Update release test docs (#13880)
* Update release test docs

* Update
2021-02-04 13:10:56 +01:00
Richard LiawandGitHub 6c77aeb98a [docs] ray slack remove banners (#13898)
Signed-off-by: Richard Liaw <rliaw@berkeley.edu>
2021-02-04 01:14:34 -08:00
0fc81e2393 [tune] fix gpu check (#13825)
Co-authored-by: Amog Kamsetty <amogkamsetty@yahoo.com>
2021-02-04 01:13:58 -08:00
Eric LiangandGitHub e79a380a7e Check in shuffle code as experimental (#13899) 2021-02-04 00:24:16 -08:00
Clark ZinzowandGitHub 243f678ffd Fall back to random port instead of default port for non-primary Redis shards; attempt to cluster Redis shard ports close to each other. (#13847) 2021-02-03 22:00:15 -08:00
Alex WuandGitHub a13208f113 Scalability envelope readme typo (#13874) 2021-02-03 21:43:45 -08:00
Tao WangandGitHub 44aa9c173f Rename timeout to period with heartbeat interval (#13872) 2021-02-04 10:37:28 +08:00
Tao WangandGitHub e0d9c8f0a8 Always replace DEL with UNLINK (#13832) 2021-02-04 10:30:00 +08:00
Dmitri GekhtmanandGitHub 1187d1dd3e [autoscaler][kubernetes][operator] Rudimentary error handling, make "MODIFIED" -> update event work. (#13756) 2021-02-03 20:07:11 -06:00
Eric LiangandGitHub e8fce9f1f3 Check Ray client protocol version (#13886)
* wip

* wip

* fix tests
2021-02-03 16:44:09 -08:00
Clark ZinzowandGitHub 407302f93a [Core] Ownership-based Object Directory - Changed infinite short-poll location subscription to long-poll. (#13841) 2021-02-03 14:16:42 -08:00
SangBin ChoandGitHub cb9fa90203 [Object Spilling] Add consumed bytes to detect thrashing. (#13853) 2021-02-03 14:16:26 -08:00
Barak MichenerandGitHub 77ee2c569f [ray_client] convert things registered for ray into ray_client (#13639) 2021-02-03 13:30:05 -08:00
f14171ced9 [Core] Put raylet ip's in resource usage report (#13871)
* .

* done?

Co-authored-by: Alex Wu <alex@anyscale.com>
2021-02-03 11:28:56 -08:00
79310452e7 Enabling the cancellation of non-actor tasks in a worker's queue 2 (#13244)
* wrote code to enable cancellation of queued non-actor tasks

* minor changes

* bug fixes

* added comments

* rev1

* linting

* making ActorSchedulingQueue::CancelTaskIfFound raise a fatal error

* bug fix

* added two unit tests

* linting

* iterating through pending_normal_tasks starting from end

* fixup! iterating through pending_normal_tasks starting from end

* fixup! fixup! iterating through pending_normal_tasks starting from end

* post merge fixes

* added debugging instructions, pulled Accept() out of guarded loop

* removed debugging instructions, linting

* first commit

* lint

* lint

* added hack to avoid race condition in test stress

* moved hack

* fix test cancel

* removed hack (hopefully no longer needed)

* Revert "removed hack (hopefully no longer needed)"

This reverts commit 99d0e7c91539f290700f50aaaed805dcde04a5ee.

* added sleep in mock_worker.cc

* sleep function fixup to work on windows

* sleep in test_fast both for force=true and force=false

* linting

Co-authored-by: Ian <ian.rodney@gmail.com>
2021-02-03 10:20:12 -08:00
Haoyuan GeandGitHub 875ea3fe1d [docs] Update actors.rst (#13873)
Add "ray.get" when calling the actor method.
2021-02-03 09:51:53 -08:00
Edward OakesandGitHub a695c651ee [serve] Small cleanups for BackendState (#13870) 2021-02-03 11:46:25 -06:00
2a903b904a [joblib] Log once the context warning argument. (#13865)
Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
2021-02-03 00:23:20 -08:00
Eric LiangandGitHub d335ce2aab Move the tune driver into a remote task (#13778) 2021-02-02 18:41:45 -08:00
fangfengbinandGitHub b4684cf37a Fix bug that otal_commands_queued_ is not initialized (#13852) 2021-02-03 10:00:15 +08:00
architkulkarniandGitHub c8e1f07c52 remove starlette install instruction (#13869) 2021-02-02 14:37:55 -08:00
architkulkarniandGitHub 32fc649f39 [serve] Add example code for custom status code response (#13868) 2021-02-02 16:30:45 -06:00
Edward OakesandGitHub fc956e084a [Hotfix] Lint (#13864) 2021-02-02 12:56:50 -08:00
JamesandGitHub 863c1b8282 Add podman support (#13633) 2021-02-02 11:09:43 -08:00
Sven MikaandGitHub 9ac731558b [RLlib] Unify fcnet initializers for the value output layer (std=1.0 in torch, but 0.01 in tf). (#13733) 2021-02-02 18:42:49 +01:00
Sven MikaandGitHub 0a0d9183fe [RLlib] Trajectory view API example script (enhancements and tf2 support). (#13786) 2021-02-02 18:42:18 +01:00
Edward OakesandGitHub a6138ca31f [serve] Support batches for ImportedBackends (#13843) 2021-02-02 09:44:01 -06:00
Kai FrickeandGitHub d29fcfb45c [tune] catch SIGINT signal and trigger experiment checkpoint (#13767)
* [tune] catch SIGINT signal and trigger experiment checkpoint

* Apply suggestions from code review

* Fix user guide docs

* Update doc/source/tune/user-guide.rst
2021-02-02 14:52:09 +01:00
Stanislav ChekmenevandGitHub b9c15a2551 [RLlib] Issue #13761: Fix get action shape (#13764) 2021-02-02 13:13:43 +01:00
Raoul KhouriandGitHub 714c367b9d [RLlib] Trainer._validate_config idempotentcy correction (issue 13427) (#13556) 2021-02-02 13:11:57 +01:00
QuantumMechaandGitHub 0c93bb77cb [RLlib] Update Documentation for Curiosity's support of continuous actions (#13784)
Only (Multi)Discrete action spaces are supported so far according to https://github.com/ray-project/ray/blob/master/rllib/utils/exploration/curiosity.py
2021-02-02 13:10:09 +01:00
Sven MikaandGitHub 52c94b7ee9 [RLlib] Allow SAC to use custom models as Q- or policy nets and deprecate "state-preprocessor" for image spaces. (#13522) 2021-02-02 13:05:58 +01:00
Eric LiangandGitHub fa4290090d Add Ray client protocol version (#13846) 2021-02-02 00:19:08 -08:00
Eric LiangandGitHub 26beb3b67b Revert "Revert "Enable Ray client server by default (#13350)" (#13429)" (#13442)
* Revert "Revert "Enable Ray client server by default (#13350)" (#13429)"

This reverts commit 560299972c.

* fix job id collision with ray client server
2021-02-02 00:17:29 -08:00
88ab887cc4 Unconditionally retry all RPC errors on client connect (#13845)
* wip

* Update python/ray/util/client/worker.py

Co-authored-by: fangfengbin <869218239a@zju.edu.cn>

Co-authored-by: fangfengbin <869218239a@zju.edu.cn>
2021-02-02 00:10:35 -08:00
Eric LiangandGitHub d71eeac2d6 remove lru evict docs (#13849) 2021-02-02 00:07:47 -08:00
SangBin ChoandGitHub 886217c333 [Object Spilling] Skip normal ray.get path when spilling objects. (#13831) 2021-02-01 16:03:34 -08:00
Eric LiangandGitHub e4d30430c0 Fix naming of ray_spilled_objects directory 2021-02-01 15:46:40 -08:00
Barak MichenerandGitHub 26ba95e96d [python/ray]: add cloudpickle dependency (#13838)
Change-Id: I248a2174c27cacb84a1cf0fd1feaa99535a90b71
2021-02-01 15:27:39 -08:00
1ee5d5faff [AWS] Fill-in AMI if not provided (#13808)
* fill in default ami if not provided

* lint fix

* quick test

* Update python/ray/tests/aws/test_autoscaler_aws.py

* Update python/ray/tests/aws/test_autoscaler_aws.py

* fix test

* fix tests

* fix lint

* remove bad test

Co-authored-by: Ameer Haj Ali <ameerh@berkeley.edu>
2021-02-01 14:30:48 -08:00
Barak MichenerandGitHub 55566bc797 [ray_client]: Add python version check and test (and some minor fixes along the way) (#13722) 2021-02-01 13:04:38 -08:00
Stephanie WangandGitHub 754bee9282 [core][object spillin] Fix bugs in admission control (#13781) 2021-02-01 10:48:21 -08:00
SongGuyangandGitHub 6e53a71978 bug fix for doc (#13834) 2021-02-01 21:13:43 +08:00
SongGuyangandGitHub 361e5f0bef support dynamic library loading in C++ worker (#13734) 2021-02-01 19:24:33 +08:00
Tao WangandGitHub 1d2ab018b0 Use right reserve size (#13829) 2021-02-01 15:49:34 +08:00
9d7b8b58a2 [autoscaler] Remove min workers from multi node type examples (#13814)
* prepare for head node

* move command runner interface outside _private

* remove space

* Eric

* flake

* min_workers in multi node type

* fixing edge cases

* eric not idle

* fix target_workers to consider min_workers of node types

* idle timeout

* minor

* minor fix

* test

* lint

* eric v2

* eric 3

* min_workers constraint before bin packing

* Update resource_demand_scheduler.py

* Revert "Update resource_demand_scheduler.py"

This reverts commit 818a63a2c86d8437b3ef21c5035d701c1d1127b5.

* reducing diff

* make get_nodes_to_launch return a dict

* merge

* weird merge fix

* auto fill instance types for AWS

* Alex/Eric

* Update doc/source/cluster/autoscaling.rst

* merge autofill and input from user

* logger.exception

* make the yaml use the default autofill

* docs Eric

* remove test_autoscaler_yaml from windows tests

* lets try changing the test a bit

* return test

* lets see

* edward

* Limit max launch concurrency

* commenting frac TODO

* move to resource demand scheduler

* use STATUS UP TO DATE

* Eric

* make logger of gc freed refs debug instead of info

* add cluster name to docker mount prefix directory

* grrR

* fix tests

* moving docker directory to sdk

* move the import to prevent circular dependency

* smallf fix

* ian

* fix max launch concurrency bug to assume failing nodes as pending and consider only load_metric's connected nodes as running

* small fix

* remove global min_workers from mult-node-type-examples

Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
Co-authored-by: Eric Liang <ekhliang@gmail.com>
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local>
2021-01-31 23:29:57 -08:00
SangBin ChoandGitHub d1ec787d9d [Object Spilling] Turn on by default. (#13745)
* Done.

* in progress.

* in progress.

* fixed tests.

* Fix.
2021-01-31 23:28:37 -08:00