wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-07-03 00:52:10 +08:00
Code Issues Packages Projects Releases Wiki Activity

[autoscaler] Improve argument handling for submit (#7986)

Browse Source 
* docs

* Apply suggestions from code review

Co-Authored-By: Kristian Hartikainen <kristian.hartikainen@gmail.com>

* ok

Co-authored-by: Kristian Hartikainen <kristian.hartikainen@gmail.com>
This commit is contained in:
Richard Liaw
2020-04-13 15:53:42 -07:00
committed by GitHub
parent e68d601ec7
commit e97adba6ac
2 changed files with 26 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/tune/guides/plot_tune-distributed.rst
+8 -9
View File
@@ -50,7 +50,7 @@ If you used a cluster configuration (starting a cluster with ``ray up`` or ``ray
.. code-block:: bash
ray submit tune-default.yaml tune_script.py --args="--ray-address=localhost:6379"
ray submit tune-default.yaml tune_script.py -- --ray-address=localhost:6379
.. tip::
@@ -77,7 +77,7 @@ If you already have a list of nodes, you can follow the local private cluster se
.. code-block:: bash
ray submit tune-default.yaml tune_script.py --args="--ray-address=localhost:6379"
ray submit tune-default.yaml tune_script.py -- --ray-address=localhost:6379
Manual Local Cluster Setup
~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -133,7 +133,7 @@ Ray currently supports AWS and GCP. Follow the instructions below to launch node
.. code-block:: bash
ray submit tune-default.yaml tune_script.py --start --args="--ray-address=localhost:6379"
ray submit tune-default.yaml tune_script.py --start -- --ray-address=localhost:6379
.. image:: /images/tune-upload.png
:scale: 50%
@@ -221,9 +221,8 @@ Here is an example for running Tune on spot instances. This assumes your AWS cre
.. code-block:: bash
ray submit tune-default.yaml mnist_pytorch_trainable.py \
--args="--ray-address=localhost:6379" \
--start
ray submit tune-default.yaml mnist_pytorch_trainable.py --start -- --ray-address=localhost:6379
4. Optionally for testing on AWS or GCP, you can use the following to kill a random worker node after all the worker nodes are up
@@ -237,7 +236,7 @@ To summarize, here are the commands to run:
wget https://raw.githubusercontent.com/ray-project/ray/master/python/ray/tune/examples/mnist_pytorch_trainable.py
wget https://raw.githubusercontent.com/ray-project/ray/master/python/ray/tune/tune-default.yaml
ray submit tune-default.yaml mnist_pytorch_trainable.py --args="--ray-address=localhost:6379" --start
ray submit tune-default.yaml mnist_pytorch_trainable.py --start -- --ray-address=localhost:6379
# wait a while until after all nodes have started
ray kill-random-node tune-default.yaml --hard
@@ -257,12 +256,12 @@ Below are some commonly used commands for submitting experiments. Please see the
# Upload `tune_experiment.py` from your local machine onto the cluster. Then,
# run `python tune_experiment.py --address=localhost:6379` on the remote machine.
$ ray submit CLUSTER.YAML tune_experiment.py --args="--address=localhost:6379"
$ ray submit CLUSTER.YAML tune_experiment.py -- --address=localhost:6379
# Start a cluster and run an experiment in a detached tmux session,
# and shut down the cluster as soon as the experiment completes.
# In `tune_experiment.py`, set `tune.run(upload_dir="s3://...")` to persist results
$ ray submit CLUSTER.YAML --tmux --start --stop tune_experiment.py --args="--address=localhost:6379"
$ ray submit CLUSTER.YAML --tmux --start --stop tune_experiment.py -- --address=localhost:6379
# To start or update your cluster:
$ ray up CLUSTER.YAML [-y]
python/ray/scripts/scripts.py
+18 -4
View File
@@ -753,9 +753,14 @@ def rsync_up(cluster_config_file, source, target, cluster_name, all_nodes):
type=int,
help="Port to forward. Use this multiple times to forward multiple ports.")
@click.argument("script", required=True, type=str)
@click.option("--args", required=False, type=str, help="Script args.")
@click.option(
"--args",
required=False,
type=str,
help="(deprecated) Use '-- --arg1 --arg2' for script args.")
@click.argument("script_args", nargs=-1)
def submit(cluster_config_file, docker, screen, tmux, stop, start,
cluster_name, port_forward, script, args):
cluster_name, port_forward, script, args, script_args):
"""Uploads and runs a script on the specified cluster.
The script is automatically synced to the following location:
@@ -763,9 +768,16 @@ def submit(cluster_config_file, docker, screen, tmux, stop, start,
os.path.join("~", os.path.basename(script))
Example:
>>> ray submit [CLUSTER.YAML] experiment.py --args="--smoke-test"
>>> ray submit [CLUSTER.YAML] experiment.py -- --smoke-test
"""
assert not (screen and tmux), "Can specify only one of `screen` or `tmux`."
assert not (script_args and args), "Use -- --arg1 --arg2 for script args."
if args:
logger.warning(
"ray submit [yaml] [script.py] --args=... is deprecated and "
"will be removed in a future version of Ray. Use "
"`ray submit [yaml] script.py -- --arg1 --arg2` instead.")
if start:
create_or_update_cluster(cluster_config_file, None, None, False, False,
@@ -775,7 +787,9 @@ def submit(cluster_config_file, docker, screen, tmux, stop, start,
rsync(cluster_config_file, script, target, cluster_name, down=False)
command_parts = ["python", target]
if args is not None:
if script_args:
command_parts += list(script_args)
elif args is not None:
command_parts += [args]
port_forward = [(port, port) for port in list(port_forward)]