* Added support for running different docker containers on clusters
* Remove node specific container names
* Keep old options and expand with node specific configuration
* Optimized imports
* Changed docker fields for autoscaler
* Auto reformat
* Updated comments
* Updated condition
* Run linter
* Updated example
* Changed condition for docker images, updated examples
* Removed duplicate line
* Fixed setup_commands
* Update autoscaler.py
* fix_better_image
Breaks the node provider node getter into cached and non-cached versions.
Fixes#3930 by updating the node label finger print before updating labels.
Fixes#3935 by refreshing node cache if node ip is not found.
- NodeUpdater gets its' IP in parallel now (no longer in __init__)
- We use persistent connections in SSH (temp folder created only for ray; ControlMaster)
- hash_runtime_conf was performing a pointless hexlify step, wasting time on large files
- We use NodeUpdaterThreads and share the NodeProvider; NodeUpdaterProcess is removed
- AWSNodeProvider caches nodes more aggressively
- NodeProvider now has a shim batch terminate_nodes() call; AWSNodeProvider parallelises it; the autoscaler uses it
- AWSNodeProvider batches EC2 update_tags calls
- Logging changes throughout to provide standardised timing information for profiling
- Pulled out a few unnecessary is_running calls (NodeUpdater will loop waiting for SSH anyway)
## Related issue number
Issue #3599
## What do these changes do?
This option goes along with `min_workers`, and `max_workers`. When the
cluster is first brought up (or when it is refreshed with a subsequent
`ray up`) this number of nodes will be started.
It's a workaround for issues of scaling (see related issues) where it
can take a long time (or forever in the case where the head node has
`--num-cpus 0`) to scale up a cluster in response to increasing demand.
## Related issue number
Workaround for https://github.com/ray-project/ray/issues/3339 and https://github.com/ray-project/ray/issues/2106
* AWS: support multiple availability zones (fix#2177)
* Bugfix: [] rather than ()
* Test config
* Test config tweaks
* Remove test config
* Formatting fixes
* Update YAML config
* Add flake8 to Travis
* Add flake8-comprehensions
[flake8 plugin](https://github.com/adamchainz/flake8-comprehensions) that
checks for useless constructions.
* Use generators instead of lists where appropriate
A lot of the builtins can take in generators instead of lists.
This commit applies `flake8-comprehensions` to find them.
* Fix lint error
* Fix some string formatting
The rest can be fixed in another PR
* Fix compound literals syntax
This should probably be merged after #1963.
* dict() -> {}
* Use dict literal syntax
dict(...) -> {...}
* Rewrite nested dicts
* Fix hanging indent
* Add missing import
* Add missing quote
* fmt
* Add missing whitespace
* rm duplicate pip install
This is already installed in another file.
* Fix indent
* move `merge_dicts` into utils
* Bring up to date with `master`
* Add automatic syntax upgrade
* rm pyupgrade
In case users want to still use it on their own, the upgrade-syn.sh script was
left in the `.travis` dir.
* Use pep8 style
The original style file is actually just pep8 style, but with everything
spelled out. It's easier to use the `based_on_style` feature. Any overrides are
clearer that way.
* Improve yapf script
1. Do formatting in parallel
2. Lint RLlib
3. Use .style.yapf file
* Pull out expressions into variables
* Don't format rllib
* Don't allow splits in dicts
* Apply yapf
* Disallow single line if-statements
* Use arithmetic comparison
* Simplify checking for changed files
* Pull out expr into var
* Fri Feb 16 13:53:50 PST 2018
* Sat Feb 17 15:32:08 PST 2018
* Sat Feb 17 15:44:59 PST 2018
* fix
* Sun Feb 18 14:46:24 PST 2018
* Sun Feb 18 14:46:37 PST 2018
* Sun Feb 18 14:55:52 PST 2018
* Sun Feb 18 15:14:32 PST 2018
* Wed Feb 21 17:34:17 PST 2018
* Sun Feb 25 17:51:17 PST 2018
* Sun Feb 25 22:18:40 PST 2018
* Wed Feb 28 13:19:05 PST 2018
* Wed Feb 28 13:22:13 PST 2018
* Wed Feb 28 13:33:29 PST 2018
* Wed Feb 28 13:35:33 PST 2018
* add ex
* Fri Mar 2 12:50:17 PST 2018
* Fri Mar 2 12:54:31 PST 2018
* Bring cloudpickle version 0.5.2 inside the repo.
* Use internal copy of cloudpickle everywhere.
* Fix linting.
* Import ordering.
* Change __init__.py.
* Set pickler in serialization context.
* Don't check ray location.
* some autoscaling config tweaks
* Sun Jan 14 13:56:55 PST 2018
* Mon Jan 15 14:21:09 PST 2018
* increase backoff
* Mon Jan 15 14:40:47 PST 2018
* check boto version
