* Basic idea
* Small fixes
* dockerize start commands in Command Runner
* Remove run_init from CommandRunnerInterface
* Add Parens
Co-authored-by: Simon Mo <simon.mo@hey.com>
* Cleaning up
* Response to richards comments
* Further small fixes
* Fix Json
* schema format fix
* cleanup
* run more often
* fix indent
* Fix richards responses
* fix ups
* remove docker_commands from schema
* default to list
* fix docker cmd runner test
* lint fix
Co-authored-by: Simon Mo <simon.mo@hey.com>
* Separate out file_mounts contents hashing into its own separate hash
Add an option to continuously sync file_mounts from head node to worker nodes:
monitor.py will re-sync file mounts whenver contents change but will only run setup_commands if the config also changes
* add test and default value for file_mounts_sync_continuously
* format code
* Update comments
* Add param to skip setup commands when only file_mounts content changed during monitor.py's update tick
Fixed so setup commands run when ray up is run and file_mounts content changes
* Refactor so that runtime_hash retains previous behavior
runtime_hash is almost identical as before this PR. It is used to determine if setup_commands need to run
file_mounts_contents_hash is an additional hash of the file_mounts content that is used to detect when only file syncing has to occur.
Note: runtime_hash value will have changed from before the PR because we hash the hash of the contents of the file_mounts as a performance optimization
* fix issue with hashing a hash
* fix bug where trying to set contents hash when it wasn't generated
* Fix lint error
Fix bug in command_runner where check_output was no longer returning the output of the command
* clear out provider between tests to get rid of flakyness
* reduce chance of race condition from node_launcher launching a node in the middle of an autoscaler.update call
