Commit Graph
26 Commits
Author SHA1 Message Date
Tao WangandGitHub 1d5694ddea [GCS]Use direct getting instead of pub-sub to update load metrics in monitor.py (#11339) 2020-10-28 11:23:18 -07:00
Alex WuandGitHub 7466ce82df [Autoscaler] Placement group autoscaling (#11243) 2020-10-14 13:11:46 -07:00
Ameer Haj AliandGitHub 06fe690682 [autoscaler] Limit max launch concurrency per node type (#11242) 2020-10-12 09:45:52 -07:00
Richard LiawandGitHub 0d93b1de93 [autoscaler] rsync to node (#11122) 2020-10-01 01:42:49 -07:00
Eric LiangandGitHub 8f73014291 [autoscaler] Make it more clear NodeProvider and CommandRunner are private APIs (#11086) 2020-09-30 16:24:37 -07:00
Tao WangandGitHub 0dcfa9ed6c Add light heartbeat flag in python and use it in load metrics (#11032) 2020-09-30 11:39:28 -07:00
Ameer Haj AliandGitHub f17ad5d060 [autoscaler] resource demand scheduler get_nodes_to_launch should return Dict not List (#11118) 2020-09-29 15:32:56 -07:00
0d36e4c025 [autoscaler] Support min_workers for multi node type (#11041)
* prepare for head node

* move command runner interface outside _private

* remove space

* Eric

* flake

* min_workers in multi node type

* fixing edge cases

* eric not idle

* fix target_workers to consider min_workers of node types

* idle timeout

* minor

* minor fix

* test

* lint

* eric v2

* eric 3

* min_workers constraint before bin packing

* Update resource_demand_scheduler.py

* Revert "Update resource_demand_scheduler.py"

This reverts commit 818a63a2c86d8437b3ef21c5035d701c1d1127b5.

* reducing diff

Co-authored-by: Ameer Haj Ali <ameerhajali@ameers-mbp.lan>
Co-authored-by: Alex Wu <alex@anyscale.io>
Co-authored-by: Alex Wu <itswu.alex@gmail.com>
2020-09-28 22:02:01 -07:00
Eric LiangandGitHub 6a227ae501 [autoscaler] Split autoscaler interface public private (#10898) 2020-09-18 18:16:23 -07:00
Alex WuandGitHub e56f2b8586 [autoscaler] hotfix calculate_node_resources (#10874) 2020-09-18 13:39:00 -07:00
Alex WuandGitHub 1ff56765f0 [Autoscaler] Fix utilization calc (#10728) 2020-09-11 12:41:08 -07:00
Ian RodneyandGitHub ea6fe0f2a1 [autoscaler] use default value (#10706) 2020-09-10 17:06:56 -07:00
Alex WuandGitHub 8906c1a59f [Autoscaler] Demand autoscaler take into account utilized resources (#10464) 2020-09-06 20:54:44 -07:00
Ian RodneyandGitHub b9633a2b67 [docker] Support multiple node types (#10504) 2020-09-02 18:27:59 -07:00
Alex WuandGitHub 23bbe0f36a [Autoscaler] Reload config (#10450) 2020-09-01 14:37:04 -07:00
Alex WuandGitHub b1f3c9e10e [Autoscaler] Fix resource passing bug fix (#10397) 2020-08-28 15:43:18 -07:00
Alex WuandGitHub 6d2af33a01 [Autoscaler] Proper resource demand plumbing (#10329) 2020-08-26 23:36:01 -07:00
Alex WuandGitHub 9ca159aa0b [Autoscaler] Multi node commands (#10236) 2020-08-25 23:35:38 -07:00
Eric LiangandGitHub 4761eacc3e [autoscaler] Also account for head node resources in multi node type autoscaling (#10230) 2020-08-24 10:26:22 -07:00
Eric LiangandGitHub 0baf992a4f [hotfix] [autoscaler] Address remaining comments on renaming instance => node (#10229)
* more renaming

* fix import
2020-08-20 14:37:41 -07:00
Eric LiangandGitHub 85a6876119 [autoscaler] Rename instance_type => node_type, TAG_RAY_INSTANCE_TYPE => TAG_RAY_USER_NODE_TYPE (#10207) 2020-08-20 12:27:11 -07:00
Eric LiangandGitHub 538cb802d5 [autoscaler] Refactor multi node type autoscaler config (#10190) 2020-08-19 20:46:00 -07:00
Alex WuandGitHub 0b5d5ec17d [Autoscaler] Pass custom resources to "ray start" multi instance autoscaling (#9986) 2020-08-17 22:34:07 -07:00
Alex WuandGitHub a54d3a1cee [Autoscaler] Run resource demand scheduler in CI (#9945)
* .

* .
2020-08-07 13:30:39 -07:00
Eric LiangandGitHub 82a5ed3fb5 [autoscaler] Instance type assert fix for non-AWS node providers (#9223) 2020-07-01 17:54:20 -07:00
Eric LiangandGitHub 536795ef79 [autoscaler] Initial support for multiple worker types (#9096)
* wip

* fix

* update

* debug state

* fix

* update

* update

* fix test

* fix

* fix

* update

* fix

* types and docs

* update
2020-06-24 23:08:30 -07:00