Ian Rodney and GitHub
39c598bab0
[docker] No Docker CP if File Mount ( #10633 )
2020-09-08 10:25:24 -07:00
f38dba09b2
[cli] Support non-interactive logging ( #10203 )
...
* add non-interactive logging
* logger
* add formatter support, add flag
* checkpoint
* format
* fix-interactive
* fixup
* fomrmat
* fix
* revert
* fix
* fix
* fix
* fix
* fx
* fix-test
* fix-commands
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-09-05 15:41:35 -07:00
Maksim Smolin and GitHub
02efa2549d
[cli] Fix colors on light themes ( #10577 )
2020-09-04 10:45:27 -07:00
Ian Rodney and GitHub
283f4d1060
[docker] Use tmp paths for rsync and fix file_mounts on docker ( #10368 )
2020-09-01 13:14:35 -07:00
Ian Rodney and GitHub
c644650818
[docker] Run docker stop in teardown_cluster ( #10407 )
2020-09-01 11:25:37 -07:00
Edward Oakes and GitHub
c3ed403def
fix typo ( #10382 )
2020-08-28 09:57:04 -05:00
Richard Liaw and GitHub
29e8a664c4
[cli] make sure old-style works ( #10344 )
2020-08-26 15:26:24 -07:00
dc378a80b7
[autoscaler/docker] Docker Inititialization Revamp ( #9515 )
...
* Basic idea
* Small fixes
* dockerize start commands in Command Runner
* Remove run_init from CommandRunnerInterface
* Add Parens
Co-authored-by: Simon Mo <simon.mo@hey.com >
* Cleaning up
* Response to richards comments
* Further small fixes
* Fix Json
* schema format fix
* cleanup
* run more often
* fix indent
* Fix richards responses
* fix ups
* remove docker_commands from schema
* default to list
* fix docker cmd runner test
* lint fix
Co-authored-by: Simon Mo <simon.mo@hey.com >
2020-08-26 10:29:06 -07:00
245c0a9e43
[cli] Tests ( #10057 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-08-22 13:29:10 -07:00
Richard Liaw and GitHub
f87669372d
[cli] enable log-new-style by default ( #10213 )
2020-08-21 15:21:43 -07:00
Eric Liang and GitHub
85a6876119
[autoscaler] Rename instance_type => node_type, TAG_RAY_INSTANCE_TYPE => TAG_RAY_USER_NODE_TYPE ( #10207 )
2020-08-20 12:27:11 -07:00
Eric Liang and GitHub
538cb802d5
[autoscaler] Refactor multi node type autoscaler config ( #10190 )
2020-08-19 20:46:00 -07:00
Richard Liaw and GitHub
2fd59de05d
[autoscaler] hotfix - swallowed error for missing yaml ( #10212 )
2020-08-19 20:02:56 -07:00
Richard Liaw and GitHub
7a56c3b71a
[cli] create_or_update_cluster fix ( #10085 )
2020-08-13 00:54:45 -07:00
40b8e35d61
[cli] New logging for the rest of the ray commands ( #9984 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-08-11 09:58:23 -07:00
0392bb7a72
[Autoscaler] Command Output Improvement ( #9699 )
...
* cross-platform prototype
* checkpoint
* Address PR comments
* format
* prepare to push
* format
* PR comments
* fixes
* fixtest
* Revert "fixtest"
This reverts commit d6f54353e1b891c784417bb8d0304c18cc5bcdd8.
* return-result
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-08-10 09:49:24 -07:00
Richard Liaw and GitHub
d3cde16163
hotfix ( #9996 )
2020-08-07 21:44:10 -07:00
Maksim Smolin and GitHub
eace94d2dc
[cli] Fix issues with logging unescaped strings ( #9960 )
2020-08-06 14:44:39 -07:00
Ameer Haj Ali and GitHub
6c9ec10540
Support not specifying an SSH key in local node provider ( #9864 )
2020-08-04 10:23:43 -05:00
Alan Guo and GitHub
3506910c5d
[autoscaler] Create worker_file_mounts config ( #9762 )
2020-07-31 14:33:27 -07:00
Alan Guo and GitHub
5831737287
Introduce file_mounts_sync_continuously cluster option ( #9544 )
...
* Separate out file_mounts contents hashing into its own separate hash
Add an option to continuously sync file_mounts from head node to worker nodes:
monitor.py will re-sync file mounts whenver contents change but will only run setup_commands if the config also changes
* add test and default value for file_mounts_sync_continuously
* format code
* Update comments
* Add param to skip setup commands when only file_mounts content changed during monitor.py's update tick
Fixed so setup commands run when ray up is run and file_mounts content changes
* Refactor so that runtime_hash retains previous behavior
runtime_hash is almost identical as before this PR. It is used to determine if setup_commands need to run
file_mounts_contents_hash is an additional hash of the file_mounts content that is used to detect when only file syncing has to occur.
Note: runtime_hash value will have changed from before the PR because we hash the hash of the contents of the file_mounts as a performance optimization
* fix issue with hashing a hash
* fix bug where trying to set contents hash when it wasn't generated
* Fix lint error
Fix bug in command_runner where check_output was no longer returning the output of the command
* clear out provider between tests to get rid of flakyness
* reduce chance of race condition from node_launcher launching a node in the middle of an autoscaler.update call
2020-07-28 00:02:08 -07:00
908c0c630a
[Autoscaler] Command Line Interface improvements ( #9322 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-07-22 12:21:44 -07:00
Philipp Moritz and GitHub
a5f4659d9f
Support ray task type checking ( #9574 )
2020-07-21 19:05:42 -07:00
1e46d4e29f
[Autoscaler] Making bootstrap config part of the node provider interface ( #9443 )
...
* supporting custom bootstrap config for external node providers
* bootstrap config
* renamed config to cluster_config
* lint
* remove 2 args from importer
* complete move of bootstrap to node_provider
* renamed provider_cls
* move imports outside functions
* lint
* Update python/ray/autoscaler/node_provider.py
Co-authored-by: Eric Liang <ekhliang@gmail.com >
* final fixes
* keeping lines to reduce diff
* lint
* lamba config
* filling in -> adding for lint
Co-authored-by: Ameer Haj Ali <ameerhajali@Ameers-MacBook-Pro.local >
Co-authored-by: Eric Liang <ekhliang@gmail.com >
2020-07-16 09:54:20 -07:00
Vasily Litvinov and GitHub
6ad13e0da8
Add ability to specify SOCKS proxy for SSH connections ( #8833 )
2020-07-13 16:10:07 -07:00
Patrick Ames and GitHub
dc51b08c36
[autoscaler] Allow users to disable the cluster config cache ( #8117 )
...
* [autoscaler] Remove autoscaler config cache.
* [autoscaler] Add flag allowing users to explicitly disable the config cache.
2020-07-09 15:47:58 -07:00
Eric Liang and GitHub
09b9b81ea4
[autoscaler] Move command runners into separate file and clean up interface. ( #9340 )
...
* cleanup
* wip
* fix imports
* fix lint
2020-07-09 15:40:56 -07:00
Ian Rodney and GitHub
079c1eaa5c
[autoscaler] Require keyword arguments ( #9256 )
2020-07-06 22:12:59 -07:00
Ian Rodney and GitHub
f1173d55e0
[autoscaler] Fix cluster_shutdown using wrong args ( #9252 )
2020-07-02 09:34:50 -07:00
Ian Rodney and GitHub
42631c5b87
[autoscaler/docker] Migrate --docker to --run-env ( #9001 )
2020-06-25 20:37:42 -07:00
Eric Liang and GitHub
536795ef79
[autoscaler] Initial support for multiple worker types ( #9096 )
...
* wip
* fix
* update
* debug state
* fix
* update
* update
* fix test
* fix
* fix
* update
* fix
* types and docs
* update
2020-06-24 23:08:30 -07:00
Eric Liang and GitHub
0ff24ec8dc
Add "ray status" debug tool for autoscaler. ( #9091 )
2020-06-24 18:22:03 -07:00
Ian Rodney and GitHub
67e049bc7b
[autoscaler/docker] Relax missing docker section ( #9105 )
2020-06-24 12:35:42 -07:00
Ameer Haj Ali and GitHub
76583d4a3a
Creating head node might take more than 50-30 seconds to show up. ( #9066 )
2020-06-21 11:12:37 -07:00
8fa584a445
[autoscaler] Run ray stop on cluster before tearing it down ( #8922 )
...
Co-authored-by: Allen Yin <allenyin@anyscale.io >
2020-06-20 17:02:34 -07:00
Alan Guo and GitHub
8f9b465968
Extract out "fillout_defaults" from other config preparation actions ( #8996 )
2020-06-17 16:40:37 -07:00
0e82f0d7c3
[autoscaler] Create Docker Command Runner (v2) ( #8840 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-06-12 16:38:38 -07:00
Ian Rodney and GitHub
b07b4f2e55
Revert "[autoscaler] Create Docker Command Runner" ( #8816 )
...
This reverts commit 54189bca5a
2020-06-06 14:21:44 -07:00
54189bca5a
[autoscaler] Create Docker Command Runner ( #8806 )
...
Co-authored-by: Richard Liaw <rliaw@berkeley.edu >
2020-06-05 17:29:27 -07:00
Alex Wu and GitHub
2c485a2598
[autoscaler] Command runner interactivity ( #7198 )
2020-06-05 17:08:38 -07:00
Eric Liang and GitHub
a24d117c68
[autoscaler] Refactor code in preparation for multi instance type support ( #8632 )
...
* wip refactor
* add util
* wip
* fix
* fix
* remove
* remove extraneous string type for sg
2020-06-03 12:53:55 -07:00
Ian Rodney and GitHub
474bbc28bf
Warn if Autoscaling-config flag not set. ( #8677 )
2020-06-03 12:21:07 -07:00
Allen and GitHub
b74eb5fce6
Capture output for commands run by the autoscaler ( #7381 )
2020-03-03 10:19:21 -08:00
Richard Liaw and GitHub
52d9189d5d
[autoscaler] port-forward for attach + redis_port ( #7145 )
...
* port-forward
* fixport
* force redis port in init mode
* test
* Update python/ray/tests/test_ray_init.py
2020-02-14 15:17:00 -08:00
Maksim Smolin and GitHub
4139e02f01
[autoscaler] Add `--all-nodes` option to rsync-up ( #7065 )
...
* Add option to sync workers to rsync-up
* Format
* Rename --sync-workers to --all-nodes
2020-02-10 16:27:59 -08:00
Alex Wu and GitHub
3f99be8dad
Add 'ray dashboard' command ( #6959 )
2020-02-10 12:55:21 -08:00
e516c50745
[autoscaler]: Kill workers if the monitor raises an exception ( #3977 )
...
Co-authored-by: CJosephides <cjosephides@gmail.com >
2020-01-23 14:12:52 -06:00
Sven and Eric Liang
60d4d5e1aa
Remove future imports ( #6724 )
...
* Remove all __future__ imports from RLlib.
* Remove (object) again from tf_run_builder.py::TFRunBuilder.
* Fix 2xLINT warnings.
* Fix broken appo_policy import (must be appo_tf_policy)
* Remove future imports from all other ray files (not just RLlib).
* Remove future imports from all other ray files (not just RLlib).
* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).
* Add two empty lines before Schedule class.
* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
Edward Oakes and GitHub
dff6017272
Fix "failed to create head node" issue ( #6304 )
...
* Fix failed to create head node issue
* comments
2019-12-02 15:22:00 -08:00
Philipp Moritz and GitHub
fc655acfee
Fix linting on master branch ( #6174 )
2019-11-16 10:02:58 -08:00
