diff --git a/doc/kubernetes/submit.yaml b/doc/kubernetes/submit.yaml index f73075857..129ad34eb 100644 --- a/doc/kubernetes/submit.yaml +++ b/doc/kubernetes/submit.yaml @@ -57,7 +57,7 @@ spec: image: rayproject/examples command: [ "/bin/bash", "-c", "--" ] args: - - "wget https://raw.githubusercontent.com/ray-project/ray/master/kubernetes/example.py && + - "wget https://raw.githubusercontent.com/ray-project/ray/master/doc/kubernetes/example.py && ray start --head --redis-port=6379 --redis-shard-ports=6380,6381 --object-manager-port=12345 --node-manager-port=12346 --node-ip-address=$MY_POD_IP && python example.py" ports: diff --git a/doc/source/deploy-on-kubernetes.rst b/doc/source/deploy-on-kubernetes.rst index 76e78239f..4f5f43ea4 100644 --- a/doc/source/deploy-on-kubernetes.rst +++ b/doc/source/deploy-on-kubernetes.rst @@ -79,7 +79,7 @@ following. One of the pods will download and run `this example script`_. -.. _`this example script`: https://github.com/ray-project/ray/tree/master/kubernetes/example.py +.. _`this example script`: https://github.com/ray-project/ray/blob/master/doc/kubernetes/example.py The script prints its output. To view the output, first find the pod name by running ``kubectl get all``. You'll see output like the following. diff --git a/doc/source/rllib-algorithms.rst b/doc/source/rllib-algorithms.rst index 7e144754e..04426d5e8 100644 --- a/doc/source/rllib-algorithms.rst +++ b/doc/source/rllib-algorithms.rst @@ -354,7 +354,7 @@ Advantage Re-Weighted Imitation Learning (MARWIL) `[paper] `__ `[implementation] `__ MARWIL is a hybrid imitation learning and policy gradient algorithm suitable for training on batched historical data. When the ``beta`` hyperparameter is set to zero, the MARWIL objective reduces to vanilla imitation learning. MARWIL requires the `offline datasets API `__ to be used. -Tuned examples: `CartPole-v0 `__ +Tuned examples: `CartPole-v0 `__ **MARWIL-specific configs** (see also `common configs `__): diff --git a/doc/source/rllib-models.rst b/doc/source/rllib-models.rst index d75798c95..0c16d2b0d 100644 --- a/doc/source/rllib-models.rst +++ b/doc/source/rllib-models.rst @@ -16,7 +16,7 @@ Built-in Models and Preprocessors RLlib picks default models based on a simple heuristic: a `vision network `__ for image observations, and a `fully connected network `__ for everything else. These models can be configured via the ``model`` config key, documented in the model `catalog `__. Note that you'll probably have to configure ``conv_filters`` if your environment observations have custom sizes, e.g., ``"model": {"dim": 42, "conv_filters": [[16, [4, 4], 2], [32, [4, 4], 2], [512, [11, 11], 1]]}`` for 42x42 observations. -In addition, if you set ``"model": {"use_lstm": true}``, then the model output will be further processed by a `LSTM cell `__. More generally, RLlib supports the use of recurrent models for its policy gradient algorithms (A3C, PPO, PG, IMPALA), and RNN support is built into its policy evaluation utilities. +In addition, if you set ``"model": {"use_lstm": true}``, then the model output will be further processed by a `LSTM cell `__. More generally, RLlib supports the use of recurrent models for its policy gradient algorithms (A3C, PPO, PG, IMPALA), and RNN support is built into its policy evaluation utilities. For preprocessors, RLlib tries to pick one of its built-in preprocessor based on the environment's observation space. Discrete observations are one-hot encoded, Atari observations downscaled, and Tuple and Dict observations flattened (these are unflattened and accessible via the ``input_dict`` parameter in custom models). Note that for Atari, RLlib defaults to using the `DeepMind preprocessors `__, which are also used by the OpenAI baselines library. diff --git a/doc/source/tune-schedulers.rst b/doc/source/tune-schedulers.rst index 62938c096..283ff8413 100644 --- a/doc/source/tune-schedulers.rst +++ b/doc/source/tune-schedulers.rst @@ -7,7 +7,7 @@ By default, Tune schedules trials in serial order with the ``FIFOScheduler`` cla tune.run( ... , scheduler=AsyncHyperBandScheduler()) -Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule `__, `HyperBand `__, and an `asynchronous version of HyperBand `__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases `__. All schedulers take in a ``metric``, which is a value returned in the result dict of your Trainable and is maximized or minimized according to ``mode``. +Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule `__, `HyperBand `__, and an `asynchronous version of HyperBand `__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases `__. All schedulers take in a ``metric``, which is a value returned in the result dict of your Trainable and is maximized or minimized according to ``mode``. Current Available Trial Schedulers: @@ -119,7 +119,7 @@ An example of this can be found in `hyperband_example.py `_ for context. +Implementation details may deviate slightly from theory but are focused on increasing usability. Note: ``R``, ``s_max``, and ``eta`` are parameters of HyperBand given by the paper. See `this post `_ for context. 1. Both ``s_max`` (representing the ``number of brackets - 1``) and ``eta``, representing the downsampling rate, are fixed. In many practical settings, ``R``, which represents some resource unit and often the number of training iterations, can be set reasonably large, like ``R >= 200``. For simplicity, assume ``eta = 3``. Varying ``R`` between ``R = 200`` and ``R = 1000`` creates a huge range of the number of trials needed to fill up all brackets.