mirror of
https://github.com/wassname/ray.git
synced 2026-07-27 11:26:41 +08:00
Fixed few broken links in docs (#5477)
* hyperband link changed * tuned_examples link fix * doc lstm link fix * kubernetes example link fix
This commit is contained in:
committed by
Richard Liaw
parent
341c6926e7
commit
0916603e61
@@ -57,7 +57,7 @@ spec:
|
||||
image: rayproject/examples
|
||||
command: [ "/bin/bash", "-c", "--" ]
|
||||
args:
|
||||
- "wget https://raw.githubusercontent.com/ray-project/ray/master/kubernetes/example.py &&
|
||||
- "wget https://raw.githubusercontent.com/ray-project/ray/master/doc/kubernetes/example.py &&
|
||||
ray start --head --redis-port=6379 --redis-shard-ports=6380,6381 --object-manager-port=12345 --node-manager-port=12346 --node-ip-address=$MY_POD_IP &&
|
||||
python example.py"
|
||||
ports:
|
||||
|
||||
@@ -79,7 +79,7 @@ following.
|
||||
|
||||
One of the pods will download and run `this example script`_.
|
||||
|
||||
.. _`this example script`: https://github.com/ray-project/ray/tree/master/kubernetes/example.py
|
||||
.. _`this example script`: https://github.com/ray-project/ray/blob/master/doc/kubernetes/example.py
|
||||
|
||||
The script prints its output. To view the output, first find the pod name by
|
||||
running ``kubectl get all``. You'll see output like the following.
|
||||
|
||||
@@ -354,7 +354,7 @@ Advantage Re-Weighted Imitation Learning (MARWIL)
|
||||
|
||||
`[paper] <http://papers.nips.cc/paper/7866-exponentially-weighted-imitation-learning-for-batched-historical-data>`__ `[implementation] <https://github.com/ray-project/ray/blob/master/rllib/agents/marwil/marwil.py>`__ MARWIL is a hybrid imitation learning and policy gradient algorithm suitable for training on batched historical data. When the ``beta`` hyperparameter is set to zero, the MARWIL objective reduces to vanilla imitation learning. MARWIL requires the `offline datasets API <rllib-offline.html>`__ to be used.
|
||||
|
||||
Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rllib/examples/cartpole-marwil.py>`__
|
||||
Tuned examples: `CartPole-v0 <https://github.com/ray-project/ray/blob/master/rllib/tuned_examples/cartpole-marwil.yaml>`__
|
||||
|
||||
**MARWIL-specific configs** (see also `common configs <rllib-training.html#common-parameters>`__):
|
||||
|
||||
|
||||
@@ -16,7 +16,7 @@ Built-in Models and Preprocessors
|
||||
|
||||
RLlib picks default models based on a simple heuristic: a `vision network <https://github.com/ray-project/ray/blob/master/rllib/models/tf/visionnet_v1.py>`__ for image observations, and a `fully connected network <https://github.com/ray-project/ray/blob/master/rllib/models/tf/fcnet_v1.py>`__ for everything else. These models can be configured via the ``model`` config key, documented in the model `catalog <https://github.com/ray-project/ray/blob/master/rllib/models/catalog.py>`__. Note that you'll probably have to configure ``conv_filters`` if your environment observations have custom sizes, e.g., ``"model": {"dim": 42, "conv_filters": [[16, [4, 4], 2], [32, [4, 4], 2], [512, [11, 11], 1]]}`` for 42x42 observations.
|
||||
|
||||
In addition, if you set ``"model": {"use_lstm": true}``, then the model output will be further processed by a `LSTM cell <https://github.com/ray-project/ray/blob/master/rllib/models/tf/lstm.py>`__. More generally, RLlib supports the use of recurrent models for its policy gradient algorithms (A3C, PPO, PG, IMPALA), and RNN support is built into its policy evaluation utilities.
|
||||
In addition, if you set ``"model": {"use_lstm": true}``, then the model output will be further processed by a `LSTM cell <https://github.com/ray-project/ray/blob/master/rllib/models/tf/lstm_v1.py>`__. More generally, RLlib supports the use of recurrent models for its policy gradient algorithms (A3C, PPO, PG, IMPALA), and RNN support is built into its policy evaluation utilities.
|
||||
|
||||
For preprocessors, RLlib tries to pick one of its built-in preprocessor based on the environment's observation space. Discrete observations are one-hot encoded, Atari observations downscaled, and Tuple and Dict observations flattened (these are unflattened and accessible via the ``input_dict`` parameter in custom models). Note that for Atari, RLlib defaults to using the `DeepMind preprocessors <https://github.com/ray-project/ray/blob/master/rllib/env/atari_wrappers.py>`__, which are also used by the OpenAI baselines library.
|
||||
|
||||
|
||||
@@ -7,7 +7,7 @@ By default, Tune schedules trials in serial order with the ``FIFOScheduler`` cla
|
||||
|
||||
tune.run( ... , scheduler=AsyncHyperBandScheduler())
|
||||
|
||||
Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule <https://research.google.com/pubs/pub46180.html>`__, `HyperBand <https://arxiv.org/abs/1603.06560>`__, and an `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`__. All schedulers take in a ``metric``, which is a value returned in the result dict of your Trainable and is maximized or minimized according to ``mode``.
|
||||
Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule <https://research.google.com/pubs/pub46180.html>`__, `HyperBand <https://arxiv.org/abs/1603.06560>`__, and an `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases <https://homes.cs.washington.edu/~jamieson/hyperband.html>`__. All schedulers take in a ``metric``, which is a value returned in the result dict of your Trainable and is maximized or minimized according to ``mode``.
|
||||
|
||||
Current Available Trial Schedulers:
|
||||
|
||||
@@ -119,7 +119,7 @@ An example of this can be found in `hyperband_example.py <https://github.com/ray
|
||||
HyperBand Implementation Details
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Implementation details may deviate slightly from theory but are focused on increasing usability. Note: ``R``, ``s_max``, and ``eta`` are parameters of HyperBand given by the paper. See `this post <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`_ for context.
|
||||
Implementation details may deviate slightly from theory but are focused on increasing usability. Note: ``R``, ``s_max``, and ``eta`` are parameters of HyperBand given by the paper. See `this post <https://homes.cs.washington.edu/~jamieson/hyperband.html>`_ for context.
|
||||
|
||||
1. Both ``s_max`` (representing the ``number of brackets - 1``) and ``eta``, representing the downsampling rate, are fixed. In many practical settings, ``R``, which represents some resource unit and often the number of training iterations, can be set reasonably large, like ``R >= 200``. For simplicity, assume ``eta = 3``. Varying ``R`` between ``R = 200`` and ``R = 1000`` creates a huge range of the number of trials needed to fill up all brackets.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user