Commit Graph

2505 Commits

Author SHA1 Message Date
Ian Rodney f940ccd6fb [autoscaler] Better explain min/max/initial worker settings (#9132) 2020-06-25 20:38:03 -07:00
Ian Rodney 42631c5b87 [autoscaler/docker] Migrate --docker to --run-env (#9001) 2020-06-25 20:37:42 -07:00
Stephanie Wang 490cddc250 [core] Refactor distributed ref counting to remove owner task ID (#9049)
* Add intended worker ID to GetObjectStatus, tests

* Remove TaskID owner_id

* lint

* Update message

* lint

* Fix build
2020-06-25 17:55:03 -07:00
krfricke 22ea8dde84 [Tune] Added XGBoost tutorial and template (#9060)
* Added XGBoost tutorial and template

* XGBoost tutorial: Cut some clutter

* Apply suggestions from code review

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Added XGboost logo

* Fixed further references

Co-authored-by: Kai Fricke <kai@anyscale.com>
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-06-25 15:59:54 -07:00
mehrdadn 07655036d2 [tune] os.replace() instead of os.rename() for cross-platform (#9141)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-25 12:30:51 -07:00
Xianyang Liu 0bfcc2e5ba [core] Better support multi-nic environments by respecting user-provided IP (#8512) 2020-06-25 14:03:12 -05:00
Alex Wu 46962f5db1 [Core] Log monitor multidriver (#8953) 2020-06-25 11:05:53 -07:00
Eric Liang 4522038259 [iter] Add .transform() function for arbitrary generator transforms (#8978) 2020-06-25 11:04:14 -07:00
henktillman 0f1d99befc [autoscaler] Remove Fingerprint checking (#9133) 2020-06-25 10:22:54 -07:00
Sven Mika 4fd8977eaf [RLlib] Minor cleanup in preparation to tf2.x support. (#9130)
* WIP.

* Fixes.

* LINT.

* Fixes.

* Fixes and LINT.

* WIP.
2020-06-25 19:01:32 +02:00
Edward Oakes aa3fd62cac [serve] Add shadow traffic API (#9106) 2020-06-25 10:55:22 -05:00
Eric Liang 536795ef79 [autoscaler] Initial support for multiple worker types (#9096)
* wip

* fix

* update

* debug state

* fix

* update

* update

* fix test

* fix

* fix

* update

* fix

* types and docs

* update
2020-06-24 23:08:30 -07:00
Eric Liang 0ff24ec8dc Add "ray status" debug tool for autoscaler. (#9091) 2020-06-24 18:22:03 -07:00
Ian Rodney 67e049bc7b [autoscaler/docker] Relax missing docker section (#9105) 2020-06-24 12:35:42 -07:00
Alex Wu c152730e4a [Core] Log output from different jobs to different drivers. (#8885)
* .

* .

* Correct now

* No interactivity errors

* format

* Filtering

* lint

* .

* No more filtering

* Removed interactivity

* .

* .

* .

* .

* .

* .

* Redirection works

* formatting

* something broken?

* .

* Works

* formatting

* redirect output

* formatting

* formatting

* Fix file descriptor leakage

* format

* .

* .

* .

* .

* .

* Refactor

* .

* Only run on job switch

* .

* cleanup

* .

* ...

* Review

* .

* .

* .

* .

* whoops

* .

* Should fix bug

* .

* .

* addressed comments

* formatting

* formatting

* Fix typo

* .

* .

* .

* .

Co-authored-by: Ubuntu <ubuntu@ip-172-31-14-33.us-west-2.compute.internal>
2020-06-23 18:45:32 -07:00
henktillman acb3280235 GCP credentials (#9052) 2020-06-23 17:11:18 -07:00
Max Fitton 7904235517 Node Info Functional Components (#9073) 2020-06-23 13:11:32 -07:00
Edward Oakes c9010eb8ad [serve] Add serve.shutdown() (#8766) 2020-06-23 13:42:03 -05:00
Simon Mo b6d425526d Move actor task submission to io service (#9093) 2020-06-23 10:07:33 -07:00
Xianyang Liu b449ece2ea [SGD] Variable worker CPU requirements (#8963) 2020-06-23 00:43:27 -07:00
Alex Wu 40c15b1ba0 [ParallelIterator] Fix for_each concurrent test cases/bugs (#8964)
* Everything works

* Update python/ray/util/iter.py

Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>

* .

* .

* removed print statements

Co-authored-by: Amog Kamsetty <amogkam@users.noreply.github.com>
2020-06-22 18:26:45 -07:00
Richard Liaw e2330ffc35 [sgd] Cleanup code from last PR (#9076) 2020-06-22 15:17:07 -07:00
Ian Rodney f76552d8db [autoscaler] Expose cache_stopped_nodes in Example YAML (#8981) 2020-06-22 15:02:45 -07:00
Ian Rodney b942bcd798 [Tune] remove whitelist from deep_copy (#8997) 2020-06-22 15:02:27 -07:00
Max Fitton 1d5c123c81 [Hotfix] Fix lint mistake accidentally pushed to master (#9077)
Co-authored-by: Max Fitton <max@semprehealth.com>
2020-06-21 19:19:07 -07:00
Richard Liaw acdd873481 [docs/sgd] Fix test failure + make slack link large (#9051) 2020-06-21 15:55:06 -07:00
Simon Mo 2b5119218e [Serve] Raise exception when _scale_replicas is infeasible (#9005) 2020-06-21 15:38:58 -07:00
Ameer Haj Ali 76583d4a3a Creating head node might take more than 50-30 seconds to show up. (#9066) 2020-06-21 11:12:37 -07:00
Vishnu Deva 432ce1be50 [tune] fix for sync_on_checkpoint bug (#9057)
* #9056 fix for sync_on_checkpoint bug

* fix for failing checks

* update help string
2020-06-21 01:07:11 -07:00
Richard Liaw e6ee39a6a3 [tune] checkpoint_dir test (#8024) 2020-06-20 17:56:24 -07:00
Allen 8fa584a445 [autoscaler] Run ray stop on cluster before tearing it down (#8922)
Co-authored-by: Allen Yin <allenyin@anyscale.io>
2020-06-20 17:02:34 -07:00
mehrdadn 981f67bfb0 Fix more Windows issues (#9011)
Co-authored-by: Mehrdad <noreply@github.com>
2020-06-19 18:51:45 -07:00
Max Fitton ca66f88b96 Sortable/Groupable Memory Dashboard (#9014) 2020-06-19 16:26:35 -05:00
Max Fitton ad09aa985c Make Dashboard Port Configurable (#8999) 2020-06-19 16:26:22 -05:00
Zhilei Chen d8a9247448 Remove gcs_service_disabled ci jobs and code (#8854) 2020-06-19 11:32:27 +08:00
Simon Mo e89b8d37ae [Serve] Fix PyTorch Tutorial (#9019) 2020-06-18 15:41:43 -07:00
mehrdadn 92f67cd2ae Add Optional Fast Build Configuration (#8925)
* Fast builds by default

* Update doc/source/development.rst

Co-authored-by: Simon Mo <xmo@berkeley.edu>

Co-authored-by: Mehrdad <noreply@github.com>
Co-authored-by: Simon Mo <xmo@berkeley.edu>
2020-06-18 14:12:12 -07:00
Edward Oakes 8a99fd205e [dashboard] Pipe resource assignments to dashboard (#8998) 2020-06-18 11:14:59 -05:00
Alan Guo 8f9b465968 Extract out "fillout_defaults" from other config preparation actions (#8996) 2020-06-17 16:40:37 -07:00
henktillman 508149b3c3 Remove redundant logger warning (#8954) 2020-06-15 21:14:58 -07:00
SangBin Cho 3d1b8c24fd [Dashboard] Dashboard pubsub hotfix. (#8944) 2020-06-15 20:38:56 -05:00
Max Fitton 4a66b6783a Logical View: Restructuring, tooltips, and QoL changes (#8916) 2020-06-15 16:09:29 -05:00
Max Fitton ddb9368f2c Display GPU Utilization in the Dashboard (#8564) 2020-06-15 15:27:44 -05:00
Richard Liaw 6c49c01837 [tune] Function API checkpointing (#8471)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-06-15 10:42:54 -07:00
Scott Graham 91e57f2e53 [azure] default workers spot instances + billing profile (#8938) 2020-06-15 10:35:20 -07:00
Jack Carreira 19cc1ae781 [docs] Tune Search: Wrong parameter name (#8927) 2020-06-13 18:01:22 -07:00
mehrdadn 101c215125 Get more tests running on Windows (#6537)
* Get rid of system() calls

* Work around '/usr/share/mini' showing up on GitHub Actions (probably due to psutil truncation)

https://github.com/ray-project/ray/runs/722480047?check_suite_focus=true

* Don't check for socket max path length on Windows

* Don't check for socket existence on Windows

* Fix race condition in Windows fate-sharing

* Work around missing .exe extension for Redis tests

* Add more tests to GitHub Actions

Co-authored-by: Mehrdad <noreply@github.com>
2020-06-12 21:32:10 -07:00
Eli Meirom 5c56760fac [tune] np.array compat for logger (#8918)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-06-12 16:39:01 -07:00
Ian Rodney 0e82f0d7c3 [autoscaler] Create Docker Command Runner (v2) (#8840)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-06-12 16:38:38 -07:00
Richard Liaw 58efec0f2b [sgd] simplify cuda visible device setting (#8775) 2020-06-12 13:53:32 -07:00