wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-20 12:40:44 +08:00
Code Issues Packages Projects Releases Wiki Activity

[tune] Tune Facelift (#2472)

Browse Source 
This PR introduces the following changes:

 * Ray Tune -> Tune 
 * [breaking] Creation of `schedulers/`, moving PBT, HyperBand into a submodule
 * [breaking] Search Algorithms now must take in experiment configurations via `add_configurations` rather through initialization
 * Support `"run": (function | class | str)` with automatic registering of trainable
 * Documentation Changes
This commit is contained in:
Richard Liaw
2018-08-19 11:00:55 -07:00
committed by GitHub
parent 78b6bfb7f9
commit 62d0698097
44 changed files with 1050 additions and 664 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/conf.py
+1 -1
View File
@@ -56,7 +56,7 @@ MOCK_MODULES = ["gym",
"ray.core.generated.TablePubsub",]
for mod_name in MOCK_MODULES:
sys.modules[mod_name] = mock.Mock()
# ray.rllib.models.action_dist.py and
# ray.rllib.models.action_dist.py and
# ray.rllib.models.lstm.py will use tf.VERSION
sys.modules["tensorflow"].VERSION = "9.9.9"
doc/source/hyperband.rst
-99
View File
@@ -1,99 +0,0 @@
HyperBand and Early Stopping
============================
Ray Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule <https://research.google.com/pubs/pub46180.html>`__, `HyperBand <https://arxiv.org/abs/1603.06560>`__, and an `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`__.
Asynchronous HyperBand
----------------------
The `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__ scheduler can be plugged in on top of an existing grid or random search. This can be done by setting the ``scheduler`` parameter of ``run_experiments``, e.g.
.. code-block:: python
run_experiments({...}, scheduler=AsyncHyperBandScheduler())
Compared to the original version of HyperBand, this implementation provides better parallelism and avoids straggler issues during eliminations. An example of this can be found in `async_hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/async_hyperband_example.py>`__. **We recommend using this over the standard HyperBand scheduler.**
.. autoclass:: ray.tune.async_hyperband.AsyncHyperBandScheduler
HyperBand
---------
.. note:: Note that the HyperBand scheduler requires your trainable to support checkpointing, which is described in `Ray Tune documentation <tune.html#trial-checkpointing>`__. Checkpointing enables the scheduler to multiplex many concurrent trials onto a limited size cluster.
Ray Tune also implements the `standard version of HyperBand <https://arxiv.org/abs/1603.06560>`__. You can use it as such:
.. code-block:: python
run_experiments({...}, scheduler=HyperBandScheduler())
An example of this can be found in `hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__. The progress of one such HyperBand run is shown below.
::
== Status ==
Using HyperBand: num_stopped=0 total_brackets=5
Round #0:
Bracket(n=5, r=100, completed=80%): {'PAUSED': 4, 'PENDING': 1}
Bracket(n=8, r=33, completed=23%): {'PAUSED': 4, 'PENDING': 4}
Bracket(n=15, r=11, completed=4%): {'RUNNING': 2, 'PAUSED': 2, 'PENDING': 11}
Bracket(n=34, r=3, completed=0%): {'RUNNING': 2, 'PENDING': 32}
Bracket(n=81, r=1, completed=0%): {'PENDING': 38}
Resources used: 4/4 CPUs, 0/0 GPUs
Result logdir: ~/ray_results/hyperband_test
PAUSED trials:
- my_class_0_height=99,width=43: PAUSED [pid=11664], 0 s, 100 ts, 97.1 rew
- my_class_11_height=85,width=81: PAUSED [pid=11771], 0 s, 33 ts, 32.8 rew
- my_class_12_height=0,width=52: PAUSED [pid=11785], 0 s, 33 ts, 0 rew
- my_class_19_height=44,width=88: PAUSED [pid=11811], 0 s, 11 ts, 5.47 rew
- my_class_27_height=96,width=84: PAUSED [pid=11840], 0 s, 11 ts, 12.5 rew
... 5 more not shown
PENDING trials:
- my_class_10_height=12,width=25: PENDING
- my_class_13_height=90,width=45: PENDING
- my_class_14_height=69,width=45: PENDING
- my_class_15_height=41,width=11: PENDING
- my_class_16_height=57,width=69: PENDING
... 81 more not shown
RUNNING trials:
- my_class_23_height=75,width=51: RUNNING [pid=11843], 0 s, 1 ts, 1.47 rew
- my_class_26_height=16,width=48: RUNNING
- my_class_31_height=40,width=10: RUNNING
- my_class_53_height=28,width=96: RUNNING
.. autoclass:: ray.tune.hyperband.HyperBandScheduler
HyperBand Implementation Details
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Implementation details may deviate slightly from theory but are focused on increasing usability. Note: ``R``, ``s_max``, and ``eta`` are parameters of HyperBand given by the paper. See `this post <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`_ for context.
1. Both ``s_max`` (representing the ``number of brackets - 1``) and ``eta``, representing the downsampling rate, are fixed. In many practical settings, ``R``, which represents some resource unit and often the number of training iterations, can be set reasonably large, like ``R >= 200``. For simplicity, assume ``eta = 3``. Varying ``R`` between ``R = 200`` and ``R = 1000`` creates a huge range of the number of trials needed to fill up all brackets.
.. image:: images/hyperband_bracket.png
On the other hand, holding ``R`` constant at ``R = 300`` and varying ``eta`` also leads to HyperBand configurations that are not very intuitive:
.. image:: images/hyperband_eta.png
The implementation takes the same configuration as the example given in the paper and exposes ``max_t``, which is not a parameter in the paper.
2. The example in the `post <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`_ to calculate ``n_0`` is actually a little different than the algorithm given in the paper. In this implementation, we implement ``n_0`` according to the paper (which is `n` in the below example):
.. image:: images/hyperband_allocation.png
3. There are also implementation specific details like how trials are placed into brackets which are not covered in the paper. This implementation places trials within brackets according to smaller bracket first - meaning that with low number of trials, there will be less early stopping.
Median Stopping Rule
--------------------
The Median Stopping Rule implements the simple strategy of stopping a trial if its performance falls below the median of other trials at similar points in time. You can set the ``scheduler`` parameter as such:
.. code-block:: python
run_experiments({...}, scheduler=MedianStoppingRule())
.. autoclass:: ray.tune.median_stopping_rule.MedianStoppingRule
doc/source/images/tune.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 58 KiB

doc/source/index.rst
+9 -8
View File
@@ -40,11 +40,11 @@ View the `codebase on GitHub`_.
Ray comes with libraries that accelerate deep learning and reinforcement learning development:
- `Ray Tune`_: Hyperparameter Optimization Framework
- `Ray RLlib`_: Scalable Reinforcement Learning
- `Tune`_: Scalable Hyperparameter Search
- `RLlib`_: Scalable Reinforcement Learning
.. _`Ray Tune`: tune.html
.. _`Ray RLlib`: rllib.html
.. _`Tune`: tune.html
.. _`RLlib`: rllib.html
.. toctree::
@@ -67,12 +67,13 @@ Ray comes with libraries that accelerate deep learning and reinforcement learnin
.. toctree::
:maxdepth: 1
:caption: Ray Tune
:caption: Tune
tune.rst
tune-config.rst
hyperband.rst
pbt.rst
tune-usage.rst
tune-schedulers.rst
tune-searchalg.rst
tune-package-ref.rst
.. toctree::
:maxdepth: 1
doc/source/pbt.rst
-33
View File
@@ -1,33 +0,0 @@
Population Based Training
=========================
Ray Tune includes a distributed implementation of `Population Based Training (PBT) <https://deepmind.com/blog/population-based-training-neural-networks>`__.
PBT Scheduler
-------------
Ray Tune's PBT scheduler can be plugged in on top of an existing grid or random search experiment. This can be enabled by setting the ``scheduler`` parameter of ``run_experiments``, e.g.
.. code-block:: python
run_experiments(
{...},
scheduler=PopulationBasedTraining(
time_attr='time_total_s',
reward_attr='mean_accuracy',
perturbation_interval=600.0,
hyperparam_mutations={
"lr": [1e-3, 5e-4, 1e-4, 5e-5, 1e-5],
"alpha": lambda: random.uniform(0.0, 1.0),
...
}))
When the PBT scheduler is enabled, each trial variant is treated as a member of the population. Periodically, top-performing trials are checkpointed (this requires your Trainable to support `checkpointing <tune.html#trial-checkpointing>`__). Low-performing trials clone the checkpoints of top performers and perturb the configurations in the hope of discovering an even better variation.
You can run this `toy PBT example <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/pbt_example.py>`__ to get an idea of how how PBT operates. When training in PBT mode, a single trial may see many different hyperparameters over its lifetime, which is recorded in its ``result.json`` file. The following figure generated by the example shows PBT discovering new hyperparams over the course of a single experiment:
.. image:: pbt.png
.. autoclass:: ray.tune.pbt.PopulationBasedTraining
doc/source/rllib-training.rst
+2 -1
View File
@@ -115,11 +115,12 @@ Here is an example of the basic usage:
checkpoint = agent.save()
print("checkpoint saved at", checkpoint)
.. note::
It's recommended that you run RLlib agents with `Tune <tune.html>`__, for easy experiment management and visualization of results. Just set ``"run": AGENT_NAME, "env": ENV_NAME`` in the experiment config.
All RLlib agents are compatible with the `Tune API <tune.html#concepts>`__. This enables them to be easily used in experiments with `Tune <tune.html>`__. For example, the following code performs a simple hyperparam sweep of PPO:
All RLlib agents are compatible with the `Tune API <tune-usage.html>`__. This enables them to be easily used in experiments with `Tune <tune.html>`__. For example, the following code performs a simple hyperparam sweep of PPO:
.. code-block:: python
doc/source/tune-config.rst
-84
View File
@@ -1,84 +0,0 @@
Experiment Configuration
========================
Experiment Setup
----------------
There are two ways to setup an experiment - one via Python and one via JSON.
The first is to create an Experiment object. You can then pass in either
a single experiment or a list of experiments to `run_experiments`, as follows:
.. code-block:: python
# Single experiment
run_experiments(Experiment(...))
# Multiple experiments
run_experiments([Experiment(...), Experiment(...), ...])
.. autoclass:: ray.tune.Experiment
An example of this can be found in `hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__.
Alternatively, you can pass in a Python dict. This uses the same fields as
the `ray.tune.Experiment`, except the experiment name is the key of the top level
dictionary.
.. code-block:: python
run_experiments({
"my_experiment_name": {
"run": "my_func",
"trial_resources": { "cpu": 1, "gpu": 0 },
"stop": { "mean_accuracy": 100 },
"config": {
"alpha": tune.grid_search([0.2, 0.4, 0.6]),
"beta": tune.grid_search([1, 2]),
},
"upload_dir": "s3://your_bucket/path",
"local_dir": "~/ray_results",
"max_failures": 2
}
})
An example of this can be found in `async_hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/async_hyperband_example.py>`__.
Trial Variant Generation
------------------------
In the above example, we specified a grid search over two parameters using the ``tune.grid_search`` helper function. Ray Tune also supports sampling parameters from user-specified lambda functions, which can be used in combination with grid search.
The following shows grid search over two nested parameters combined with random sampling from two lambda functions. Note that the value of ``beta`` depends on the value of ``alpha``, which is represented by referencing ``spec.config.alpha`` in the lambda function. This lets you specify conditional parameter distributions.
.. code-block:: python
"config": {
"alpha": lambda spec: np.random.uniform(100),
"beta": lambda spec: spec.config.alpha * np.random.normal(),
"nn_layers": [
tune.grid_search([16, 64, 256]),
tune.grid_search([16, 64, 256]),
],
},
"repeat": 10,
By default, each random variable and grid search point is sampled once. To take multiple random samples or repeat grid search runs, add ``repeat: N`` to the experiment config. E.g. in the above, ``"repeat": 10`` repeats the 3x3 grid search 10 times, for a total of 90 trials, each with randomly sampled values of ``alpha`` and ``beta``.
.. note::
Lambda functions will be evaluated during trial variant generation. If you need to pass a literal function in your config, use ``tune.function(...)`` to escape it.
For more information on variant generation, see `basic_variant.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/suggest/basic_variant.py>`__.
Resource Allocation
-------------------
Ray Tune runs each trial as a Ray actor, allocating the specified GPU and CPU ``trial_resources`` to each actor (defaulting to 1 CPU per trial). A trial will not be scheduled unless at least that amount of resources is available in the cluster, preventing the cluster from being overloaded.
If GPU resources are not requested, the ``CUDA_VISIBLE_DEVICES`` environment variable will be set as empty, disallowing GPU access.
Otherwise, it will be set to the GPUs in the list (this is managed by Ray).
If your trainable function / class creates further Ray actors or tasks that also consume CPU / GPU resources, you will also want to set ``extra_cpu`` or ``extra_gpu`` to reserve extra resource slots for the actors you will create. For example, if a trainable class requires 1 GPU itself, but will launch 4 actors each using another GPU, then it should set ``"gpu": 1, "extra_gpu": 4``.
doc/source/tune-package-ref.rst
+33
View File
@@ -0,0 +1,33 @@
Tune Package Reference
=======================
ray.tune
--------
.. automodule:: ray.tune
:members:
:exclude-members: TuneError, Trainable
.. autoclass:: ray.tune.Trainable
:members:
:private-members:
ray.tune.schedulers
-------------------
.. automodule:: ray.tune.schedulers
:members:
:show-inheritance:
ray.tune.suggest
----------------
.. automodule:: ray.tune.suggest
:members:
:exclude-members: function, grid_search, SuggestionAlgorithm
:show-inheritance:
.. autoclass:: ray.tune.suggest.SuggestionAlgorithm
:members:
:private-members:
:show-inheritance:
doc/source/tune-schedulers.rst
+150
View File
@@ -0,0 +1,150 @@
Tune Trial Schedulers
=====================
By default, Tune schedules trials in serial order with the ``FIFOScheduler`` class. However, you can also specify a custom scheduling algorithm that can early stop trials or perturb parameters.
.. code-block:: python
tune.run_experiments({...}, scheduler=AsyncHyperBandScheduler())
Tune includes distributed implementations of early stopping algorithms such as `Median Stopping Rule <https://research.google.com/pubs/pub46180.html>`__, `HyperBand <https://arxiv.org/abs/1603.06560>`__, and an `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__. These algorithms are very resource efficient and can outperform Bayesian Optimization methods in `many cases <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`__. Currently, all schedulers take in a ``reward_attr``, which is assumed to be maximized.
Current Available Trial Schedulers:
.. contents::
:local:
:backlinks: none
Population Based Training (PBT)
-------------------------------
Tune includes a distributed implementation of `Population Based Training (PBT) <https://deepmind.com/blog/population-based-training-neural-networks>`__. This can be enabled by setting the ``scheduler`` parameter of ``run_experiments``, e.g.
.. code-block:: python
pbt_scheduler = PopulationBasedTraining(
time_attr='time_total_s',
reward_attr='mean_accuracy',
perturbation_interval=600.0,
hyperparam_mutations={
"lr": [1e-3, 5e-4, 1e-4, 5e-5, 1e-5],
"alpha": lambda: random.uniform(0.0, 1.0),
...
})
run_experiments({...}, scheduler=pbt_scheduler)
When the PBT scheduler is enabled, each trial variant is treated as a member of the population. Periodically, top-performing trials are checkpointed (this requires your Trainable to support `checkpointing <tune-usage.html#trial-checkpointing>`__). Low-performing trials clone the checkpoints of top performers and perturb the configurations in the hope of discovering an even better variation.
You can run this `toy PBT example <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/pbt_example.py>`__ to get an idea of how how PBT operates. When training in PBT mode, a single trial may see many different hyperparameters over its lifetime, which is recorded in its ``result.json`` file. The following figure generated by the example shows PBT discovering new hyperparams over the course of a single experiment:
.. image:: pbt.png
.. autoclass:: ray.tune.schedulers.PopulationBasedTraining
:noindex:
Asynchronous HyperBand
----------------------
The `asynchronous version of HyperBand <https://openreview.net/forum?id=S1Y7OOlRZ>`__ scheduler can be used by setting the ``scheduler`` parameter of ``run_experiments``, e.g.
.. code-block:: python
async_hb_scheduler = AsyncHyperBandScheduler(
time_attr='training_iteration',
reward_attr='episode_reward_mean',
max_t=100,
grace_period=10,
reduction_factor=3,
brackets=3)
run_experiments({...}, scheduler=async_hb_scheduler)
Compared to the original version of HyperBand, this implementation provides better parallelism and avoids straggler issues during eliminations. An example of this can be found in `async_hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/async_hyperband_example.py>`__. **We recommend using this over the standard HyperBand scheduler.**
.. autoclass:: ray.tune.schedulers.AsyncHyperBandScheduler
:noindex:
HyperBand
---------
.. note:: Note that the HyperBand scheduler requires your trainable to support checkpointing, which is described in `Tune User Guide <tune-usage.html#trial-checkpointing>`__. Checkpointing enables the scheduler to multiplex many concurrent trials onto a limited size cluster.
Tune also implements the `standard version of HyperBand <https://arxiv.org/abs/1603.06560>`__. You can use it as such:
.. code-block:: python
run_experiments({...}, scheduler=HyperBandScheduler())
An example of this can be found in `hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__. The progress of one such HyperBand run is shown below.
::
== Status ==
Using HyperBand: num_stopped=0 total_brackets=5
Round #0:
Bracket(n=5, r=100, completed=80%): {'PAUSED': 4, 'PENDING': 1}
Bracket(n=8, r=33, completed=23%): {'PAUSED': 4, 'PENDING': 4}
Bracket(n=15, r=11, completed=4%): {'RUNNING': 2, 'PAUSED': 2, 'PENDING': 11}
Bracket(n=34, r=3, completed=0%): {'RUNNING': 2, 'PENDING': 32}
Bracket(n=81, r=1, completed=0%): {'PENDING': 38}
Resources used: 4/4 CPUs, 0/0 GPUs
Result logdir: ~/ray_results/hyperband_test
PAUSED trials:
- my_class_0_height=99,width=43: PAUSED [pid=11664], 0 s, 100 ts, 97.1 rew
- my_class_11_height=85,width=81: PAUSED [pid=11771], 0 s, 33 ts, 32.8 rew
- my_class_12_height=0,width=52: PAUSED [pid=11785], 0 s, 33 ts, 0 rew
- my_class_19_height=44,width=88: PAUSED [pid=11811], 0 s, 11 ts, 5.47 rew
- my_class_27_height=96,width=84: PAUSED [pid=11840], 0 s, 11 ts, 12.5 rew
... 5 more not shown
PENDING trials:
- my_class_10_height=12,width=25: PENDING
- my_class_13_height=90,width=45: PENDING
- my_class_14_height=69,width=45: PENDING
- my_class_15_height=41,width=11: PENDING
- my_class_16_height=57,width=69: PENDING
... 81 more not shown
RUNNING trials:
- my_class_23_height=75,width=51: RUNNING [pid=11843], 0 s, 1 ts, 1.47 rew
- my_class_26_height=16,width=48: RUNNING
- my_class_31_height=40,width=10: RUNNING
- my_class_53_height=28,width=96: RUNNING
.. autoclass:: ray.tune.schedulers.HyperBandScheduler
:noindex:
HyperBand Implementation Details
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Implementation details may deviate slightly from theory but are focused on increasing usability. Note: ``R``, ``s_max``, and ``eta`` are parameters of HyperBand given by the paper. See `this post <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`_ for context.
1. Both ``s_max`` (representing the ``number of brackets - 1``) and ``eta``, representing the downsampling rate, are fixed. In many practical settings, ``R``, which represents some resource unit and often the number of training iterations, can be set reasonably large, like ``R >= 200``. For simplicity, assume ``eta = 3``. Varying ``R`` between ``R = 200`` and ``R = 1000`` creates a huge range of the number of trials needed to fill up all brackets.
.. image:: images/hyperband_bracket.png
On the other hand, holding ``R`` constant at ``R = 300`` and varying ``eta`` also leads to HyperBand configurations that are not very intuitive:
.. image:: images/hyperband_eta.png
The implementation takes the same configuration as the example given in the paper and exposes ``max_t``, which is not a parameter in the paper.
2. The example in the `post <https://people.eecs.berkeley.edu/~kjamieson/hyperband.html>`_ to calculate ``n_0`` is actually a little different than the algorithm given in the paper. In this implementation, we implement ``n_0`` according to the paper (which is `n` in the below example):
.. image:: images/hyperband_allocation.png
3. There are also implementation specific details like how trials are placed into brackets which are not covered in the paper. This implementation places trials within brackets according to smaller bracket first - meaning that with low number of trials, there will be less early stopping.
Median Stopping Rule
--------------------
The Median Stopping Rule implements the simple strategy of stopping a trial if its performance falls below the median of other trials at similar points in time. You can set the ``scheduler`` parameter as such:
.. code-block:: python
run_experiments({...}, scheduler=MedianStoppingRule())
.. autoclass:: ray.tune.schedulers.MedianStoppingRule
:noindex:
doc/source/tune-searchalg.rst
+70
View File
@@ -0,0 +1,70 @@
Tune Search Algorithms
======================
Tune provides various hyperparameter search algorithms to efficiently optimize your model. Tune allows you to use different search algorithms in combination with different trial schedulers. Tune will by default implicitly use the Variant Generation algorithm to create trials.
You can utilize these search algorithms as follows:
.. code-block:: python
run_experiments(experiments, search_alg=SearchAlgorithm(...))
Currently, Tune offers the following search algorithms:
- `Grid Search and Random Search <tune-searchalg.html#variant-generation-grid-search-random-search>`__
- `HyperOpt <tune-searchalg.html#hyperopt-search-tree-structured-parzen-estimators>`__
Variant Generation (Grid Search/Random Search)
----------------------------------------------
By default, Tune uses the `default search space and variant generation process <tune-usage.html#tune-search-space-default>`__ to create and queue trials. This supports random search and grid search as specified by the ``config`` parameter of the Experiment.
.. autoclass:: ray.tune.suggest.BasicVariantGenerator
:show-inheritance:
:noindex:
HyperOpt Search (Tree-structured Parzen Estimators)
---------------------------------------------------
The ``HyperOptSearch`` is a SearchAlgorithm that is backed by `HyperOpt <http://hyperopt.github.io/hyperopt>`__ to perform sequential model-based hyperparameter optimization.
In order to use this search algorithm, you will need to install HyperOpt via the following command:
.. code-block:: bash
$ pip install --upgrade git+git://github.com/hyperopt/hyperopt.git
This algorithm requires using the `HyperOpt search space specification <https://github.com/hyperopt/hyperopt/wiki/FMin>`__. You can use HyperOptSearch like follows:
.. code-block:: python
run_experiments(experiment_config, search_alg=HyperOptSearch(hyperopt_space, ... ))
An example of this can be found in `hyperopt_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py>`__.
.. autoclass:: ray.tune.suggest.HyperOptSearch
:show-inheritance:
:noindex:
Contributing a New Algorithm
----------------------------
If you are interested in implementing or contributing a new Search Algorithm, the API is straightforward:
.. autoclass:: ray.tune.suggest.SearchAlgorithm
:members:
:noindex:
Model-Based Suggestion Algorithms
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Often times, hyperparameter search algorithms are model-based and may be quite simple to implement. For this, one can extend the following abstract class and implement ``on_trial_result``, ``on_trial_complete``, and ``_suggest``. The abstract class will take care of Tune-specific boilerplate such as creating Trials and queuing trials:
.. autoclass:: ray.tune.suggest.SuggestionAlgorithm
:show-inheritance:
:noindex:
.. automethod:: ray.tune.suggest.SuggestionAlgorithm._suggest
:noindex:
doc/source/tune-usage.rst
+365
View File
@@ -0,0 +1,365 @@
Tune User Guide
===============
Tune Overview
-------------
.. image:: images/tune-api.svg
Tune schedules a number of *trials* in a cluster. Each trial runs a user-defined Python function or class and is parameterized either by a *config* variation from Tune's Variant Generator or a user-specified **search algorithm**. The trials are scheduled and managed by a **trial scheduler**.
More information about Tune's `search algorithms can be found here <tune-searchalg.html>`__.
More information about Tune's `trial schedulers can be found here <tune-schedulers.html>`__.
Start by installing, importing, and initializing Ray.
.. code-block:: python
import ray
import ray.tune as tune
ray.init()
Tune provides a ``run_experiments`` function that generates and runs the trials as described by the `experiment specification <tune-usage.html#experiment-configuration>`__.
.. autofunction:: ray.tune.run_experiments
:noindex:
This function will report status on the command line until all Trials stop:
::
== Status ==
Using FIFO scheduling algorithm.
Resources used: 4/8 CPUs, 0/0 GPUs
Result logdir: ~/ray_results/my_experiment
- train_func_0_lr=0.2,momentum=1: RUNNING [pid=6778], 209 s, 20604 ts, 7.29 acc
- train_func_1_lr=0.4,momentum=1: RUNNING [pid=6780], 208 s, 20522 ts, 53.1 acc
- train_func_2_lr=0.6,momentum=1: TERMINATED [pid=6789], 21 s, 2190 ts, 100 acc
- train_func_3_lr=0.2,momentum=2: RUNNING [pid=6791], 208 s, 41004 ts, 8.37 acc
- train_func_4_lr=0.4,momentum=2: RUNNING [pid=6800], 209 s, 41204 ts, 70.1 acc
- train_func_5_lr=0.6,momentum=2: TERMINATED [pid=6809], 10 s, 2164 ts, 100 acc
Experiment Configuration
------------------------
Specifying Experiments
~~~~~~~~~~~~~~~~~~~~~~
There are two ways to specify the configuration for an experiment - one via Python and one via JSON.
**Using Python**: specify a configuration is to create an Experiment object.
.. autoclass:: ray.tune.Experiment
:noindex:
An example of this can be found in `hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__.
**Using JSON/Dict**: This uses the same fields as the ``ray.tune.Experiment``, except the experiment name is the key of the top level
dictionary. Tune will convert the dict into an ``ray.tune.Experiment`` object.
.. code-block:: python
experiment_spec = {
"my_experiment_name": {
"run": my_func,
"stop": { "mean_accuracy": 100 },
"config": {
"alpha": tune.grid_search([0.2, 0.4, 0.6]),
"beta": tune.grid_search([1, 2]),
},
"trial_resources": { "cpu": 1, "gpu": 0 },
"repeat": 10,
"local_dir": "~/ray_results",
"upload_dir": "s3://your_bucket/path",
"checkpoint_freq": 10,
"max_failures": 2
}
}
run_experiments(experiment_spec)
An example of this can be found in `async_hyperband_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/async_hyperband_example.py>`__.
Model API
~~~~~~~~~
You can either pass in a Python function or Python class for model training as follows, each requiring a specific signature/interface:
.. code-block:: python
:emphasize-lines: 3,8
experiment_spec = {
"my_experiment_name": {
"run": my_trainable
}
}
# or with the Experiment API
experiment_spec = Experiment("my_experiment_name", my_trainable)
run_experiments(experiments=experiment_spec)
**Python functions** will need to have the following signature:
.. code-block:: python
def trainable(config, reporter):
"""
Args:
config (dict): Parameters provided from the search algorithm
or variant generation.
reporter (Reporter): Handle to report intermediate metrics to Tune.
"""
Tune will run this function on a separate thread in a Ray actor process. Note that trainable functions are not checkpointable, since they never return control back to their caller. See `Trial Checkpointing for more details <tune-usage.html#trial-checkpointing>`__.
.. note::
If you have a lambda function that you want to train, you will need to first register the function: ``tune.register_trainable("lambda_id", lambda x: ...)``. You can then use ``lambda_id`` in place of ``my_trainable``.
**Python classes** passed into Tune will need to subclass ``ray.tune.Trainable``.
.. autoclass:: ray.tune.Trainable
:members: __init__, _save, _restore, _train, _setup, _stop
:noindex:
Tune Search Space (Default)
~~~~~~~~~~~~~~~~~~~~~~~~~~~
You can use ``tune.grid_search`` to specify an axis of a grid search. By default, Tune also supports sampling parameters from user-specified lambda functions, which can be used independently or in combination with grid search.
The following shows grid search over two nested parameters combined with random sampling from two lambda functions, generating 9 different trials. Note that the value of ``beta`` depends on the value of ``alpha``, which is represented by referencing ``spec.config.alpha`` in the lambda function. This lets you specify conditional parameter distributions.
.. code-block:: python
:emphasize-lines: 4-11
run_experiments({
"my_experiment_name": {
"run": my_trainable,
"config": {
"alpha": lambda spec: np.random.uniform(100),
"beta": lambda spec: spec.config.alpha * np.random.normal(),
"nn_layers": [
tune.grid_search([16, 64, 256]),
tune.grid_search([16, 64, 256]),
],
}
}
})
.. note::
Lambda functions will be evaluated during trial variant generation. If you need to pass a literal function in your config, use ``tune.function(...)`` to escape it.
.. warning::
If you specify a Search Algorithm, you may not be able to use this feature, as the algorithm may require a different search space declaration.
For more information on variant generation, see `basic_variant.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/suggest/basic_variant.py>`__.
Sampling Multiple Times
~~~~~~~~~~~~~~~~~~~~~~~
By default, each random variable and grid search point is sampled once. To take multiple random samples or repeat grid search runs, add ``repeat: N`` to the experiment config.
.. code-block:: python
:emphasize-lines: 12
run_experiments({
"my_experiment_name": {
"run": my_trainable,
"config": {
"alpha": lambda spec: np.random.uniform(100),
"beta": lambda spec: spec.config.alpha * np.random.normal(),
"nn_layers": [
tune.grid_search([16, 64, 256]),
tune.grid_search([16, 64, 256]),
],
},
"repeat": 10
}
})
E.g. in the above, ``"repeat": 10`` repeats the 3x3 grid search 10 times, for a total of 90 trials, each with randomly sampled values of ``alpha`` and ``beta``.
Using GPUs (Resource Allocation)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Tune will allocate the specified GPU and CPU ``trial_resources`` to each individual trial (defaulting to 1 CPU per trial). Under the hood, Tune runs each trial as a Ray actor, using Ray's resource handling to allocate resources and place actors. A trial will not be scheduled unless at least that amount of resources is available in the cluster, preventing the cluster from being overloaded.
If GPU resources are not requested, the ``CUDA_VISIBLE_DEVICES`` environment variable will be set as empty, disallowing GPU access.
Otherwise, it will be set to the GPUs in the list (this is managed by Ray).
If your trainable function / class creates further Ray actors or tasks that also consume CPU / GPU resources, you will also want to set ``extra_cpu`` or ``extra_gpu`` to reserve extra resource slots for the actors you will create. For example, if a trainable class requires 1 GPU itself, but will launch 4 actors each using another GPU, then it should set ``"gpu": 1, "extra_gpu": 4``.
.. code-block:: python
:emphasize-lines: 4-8
run_experiments({
"my_experiment_name": {
"run": my_trainable,
"trial_resources": {
"cpu": 1,
"gpu": 1,
"extra_gpu": 4
}
}
})
Trial Checkpointing
~~~~~~~~~~~~~~~~~~~
To enable checkpointing, you must implement a `Trainable class <tune-usage.html#model-api>`__ (Trainable functions are not checkpointable, since they never return control back to their caller). The easiest way to do this is to subclass the pre-defined ``Trainable`` class and implement its ``_train``, ``_save``, and ``_restore`` abstract methods `(example) <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__. Implementing this interface is required to support resource multiplexing in Trial Schedulers such as HyperBand and PBT.
For TensorFlow model training, this would look something like this `(full tensorflow example) <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/tune_mnist_ray_hyperband.py>`__:
.. code-block:: python
class MyClass(Trainable):
def _setup(self):
self.saver = tf.train.Saver()
self.sess = ...
self.iteration = 0
def _train(self):
self.sess.run(...)
self.iteration += 1
def _save(self, checkpoint_dir):
return self.saver.save(
self.sess, checkpoint_dir + "/save",
global_step=self.iteration)
def _restore(self, path):
return self.saver.restore(self.sess, path)
Additionally, checkpointing can be used to provide fault-tolerance for experiments. This can be enabled by setting ``checkpoint_freq: N`` and ``max_failures: M`` to checkpoint trials every *N* iterations and recover from up to *M* crashes per trial, e.g.:
.. code-block:: python
:emphasize-lines: 4,5
run_experiments({
"my_experiment_name": {
"run": my_trainable
"checkpoint_freq": 10,
"max_failures": 5,
},
})
Handling Large Datasets
-----------------------
You often will want to compute a large object (e.g., training data, model weights) on the driver and use that object within each trial. Tune provides a ``pin_in_object_store`` utility function that can be used to broadcast such large objects. Objects pinned in this way will never be evicted from the Ray object store while the driver process is running, and can be efficiently retrieved from any task via ``get_pinned_object``.
.. code-block:: python
import ray
from ray.tune import run_experiments
from ray.tune.util import pin_in_object_store, get_pinned_object
import numpy as np
ray.init()
# X_id can be referenced in closures
X_id = pin_in_object_store(np.random.random(size=100000000))
def f(config, reporter):
X = get_pinned_object(X_id)
# use X
run_experiments({
"my_experiment_name": {
"run": f
}
})
Logging and Visualizing Results
-------------------------------
All results reported by the trainable will be logged locally to a unique directory per experiment, e.g. ``~/ray_results/my_experiment`` in the above example. On a cluster, incremental results will be synced to local disk on the head node. The log records are compatible with a number of visualization tools:
To visualize learning in tensorboard, install TensorFlow:
.. code-block:: bash
$ pip install tensorflow
Then, after you run a experiment, you can visualize your experiment with TensorBoard by specifying the output directory of your results. Note that if you running Ray on a remote cluster, you can forward the tensorboard port to your local machine through SSH using ``ssh -L 6006:localhost:6006 <address>``:
.. code-block:: bash
$ tensorboard --logdir=~/ray_results/my_experiment
.. image:: ray-tune-tensorboard.png
To use rllab's VisKit (you may have to install some dependencies), run:
.. code-block:: bash
$ git clone https://github.com/rll/rllab.git
$ python rllab/rllab/viskit/frontend.py ~/ray_results/my_experiment
.. image:: ray-tune-viskit.png
Finally, to view the results with a `parallel coordinates visualization <https://en.wikipedia.org/wiki/Parallel_coordinates>`__, open `ParallelCoordinatesVisualization.ipynb <https://github.com/ray-project/ray/blob/master/python/ray/tune/ParallelCoordinatesVisualization.ipynb>`__ as follows and run its cells:
.. code-block:: bash
$ cd $RAY_HOME/python/ray/tune
$ jupyter-notebook ParallelCoordinatesVisualization.ipynb
.. image:: ray-tune-parcoords.png
Client API
----------
You can modify an ongoing experiment by adding or deleting trials using the Tune Client API. To do this, verify that you have the ``requests`` library installed:
.. code-block:: bash
$ pip install requests
To use the Client API, you can start your experiment with ``with_server=True``:
.. code-block:: python
run_experiments({...}, with_server=True, server_port=4321)
Then, on the client side, you can use the following class. The server address defaults to ``localhost:4321``. If on a cluster, you may want to forward this port (e.g. ``ssh -L <local_port>:localhost:<remote_port> <address>``) so that you can use the Client on your local machine.
.. autoclass:: ray.tune.web_server.TuneClient
:members:
For an example notebook for using the Client API, see the `Client API Example <https://github.com/ray-project/ray/tree/master/python/ray/tune/TuneClient.ipynb>`__.
Examples
--------
You can find a comprehensive of examples `using Tune and its various features here <https://github.com/ray-project/ray/tree/master/python/ray/tune/examples>`__, including examples using Keras, TensorFlow, and Population-Based Training.
Further Questions or Issues?
----------------------------
You can post questions or issues or feedback through the following channels:
1. `Our Mailing List`_: For discussions about development, questions about
usage, or any general questions and feedback.
2. `GitHub Issues`_: For bug reports and feature requests.
.. _`Our Mailing List`: https://groups.google.com/forum/#!forum/ray-dev
.. _`GitHub Issues`: https://github.com/ray-project/ray/issues
doc/source/tune.rst
+70 -259
View File
@@ -1,8 +1,36 @@
Ray Tune: Hyperparameter Optimization Framework
===============================================
Tune: Scalable Hyperparameter Search
====================================
Ray Tune is a scalable hyperparameter optimization framework for reinforcement learning and deep learning. Go from running one experiment on a single machine to running on a large cluster with efficient search algorithms without changing your code.
.. image:: images/tune.png
:scale: 30%
:align: center
Tune is a scalable framework for hyperparameter search with a focus on deep learning and deep reinforcement learning.
You can find the code for Tune `here on GitHub <https://github.com/ray-project/ray/tree/master/python/ray/tune>`__.
Features
--------
* Supports any deep learning framework, including PyTorch, Tensorflow, and Keras.
* Choose among scalable hyperparameter and model search techniques such as:
- `Population Based Training (PBT) <tune-schedulers.html#population-based-training-pbt>`__
- `Median Stopping Rule <tune-schedulers.html#median-stopping-rule>`__
- `HyperBand <tune-schedulers.html#asynchronous-hyperband>`__
* Mix and match different hyperparameter optimization approaches - such as using `HyperOpt with HyperBand`_.
* Visualize results with `TensorBoard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__, `parallel coordinates (Plot.ly) <https://plot.ly/python/parallel-coordinates-plot/>`__, and `rllab's VisKit <https://media.readthedocs.org/pdf/rllab/latest/rllab.pdf>`__.
* Scale to running on a large distributed cluster without changing your code.
* Parallelize training for models with GPU requirements or algorithms that may themselves be parallel and distributed, using Tune's `resource-aware scheduling <tune-usage.html#using-gpus-resource-allocation>`__,
Take a look at `the User Guide <tune-usage.html>`__ for a comprehensive overview on how to use Tune's features.
Getting Started
---------------
@@ -10,288 +38,71 @@ Getting Started
Installation
~~~~~~~~~~~~
You'll need to first `install ray <installation.html>`__ to import Ray Tune.
You'll need to first `install ray <installation.html>`__ to import Tune.
.. code-block:: bash
pip install ray
Quick Start
~~~~~~~~~~~
This example runs a small grid search over a neural network training function using Tune, reporting status on the command line until the stopping condition of ``mean_accuracy >= 99`` is reached. Tune works with any deep learning framework.
Tune uses Ray as a backend, so we will first import and initialize Ray.
.. code-block:: python
import ray
import ray.tune as tune
ray.init()
tune.register_trainable("train_func", train_func)
all_trials = tune.run_experiments({
"my_experiment": {
"run": "train_func",
"stop": {"mean_accuracy": 99},
"config": {
"lr": tune.grid_search([0.2, 0.4, 0.6]),
"momentum": tune.grid_search([0.1, 0.2]),
}
}
})
For the function you wish to tune, add a two-line modification (note that we use PyTorch as an example but Ray Tune works with any deep learning framework):
For the function you wish to tune, pass in a ``reporter`` object:
.. code-block:: python
:emphasize-lines: 1,14
:emphasize-lines: 1,9
def train_func(config, reporter): # add a reporter arg
model = NeuralNet()
optimizer = torch.optim.SGD(
model.parameters(), lr=config["lr"], momentum=config["momentum"])
model = ( ... )
optimizer = SGD(model.parameters(),
momentum=config["momentum"])
dataset = ( ... )
for idx, (data, target) in enumerate(dataset):
# ...
output = model(data)
loss = F.MSELoss(output, target)
loss.backward()
optimizer.step()
accuracy = eval_accuracy(...)
reporter(timesteps_total=idx, mean_accuracy=accuracy) # report metrics
accuracy = model.fit(data, target)
reporter(mean_accuracy=accuracy) # report metrics
This PyTorch script runs a small grid search over the ``train_func`` function using Ray Tune, reporting status on the command line until the stopping condition of ``mean_accuracy >= 99`` is reached:
::
== Status ==
Using FIFO scheduling algorithm.
Resources used: 4/8 CPUs, 0/0 GPUs
Result logdir: ~/ray_results/my_experiment
- train_func_0_lr=0.2,momentum=1: RUNNING [pid=6778], 209 s, 20604 ts, 7.29 acc
- train_func_1_lr=0.4,momentum=1: RUNNING [pid=6780], 208 s, 20522 ts, 53.1 acc
- train_func_2_lr=0.6,momentum=1: TERMINATED [pid=6789], 21 s, 2190 ts, 100 acc
- train_func_3_lr=0.2,momentum=2: RUNNING [pid=6791], 208 s, 41004 ts, 8.37 acc
- train_func_4_lr=0.4,momentum=2: RUNNING [pid=6800], 209 s, 41204 ts, 70.1 acc
- train_func_5_lr=0.6,momentum=2: TERMINATED [pid=6809], 10 s, 2164 ts, 100 acc
In order to report incremental progress, ``train_func`` periodically calls the ``reporter`` function passed in by Ray Tune to return the current timestep and other metrics. Incremental results will be synced to local disk on the head node of the cluster.
`tune.run_experiments <tune.html#ray.tune.run_experiments>`__ returns a list of Trial objects which you can inspect results of via ``trial.last_result``.
Learn more `about specifying experiments <tune-config.html>`__.
Features
--------
Ray Tune has the following features:
- Scalable implementations of search execution techniques such as `Population Based Training (PBT) <pbt.html>`__, `Median Stopping Rule <hyperband.html#median-stopping-rule>`__, and `HyperBand <hyperband.html>`__.
- The ability to combine search execution and search algorithms, such as Model-Based Optimization (HyperOpt) with HyperBand.
- Integration with visualization tools such as `TensorBoard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__, `rllab's VisKit <https://media.readthedocs.org/pdf/rllab/latest/rllab.pdf>`__, and a `parallel coordinates visualization <https://en.wikipedia.org/wiki/Parallel_coordinates>`__.
- Flexible trial variant generation, including grid search, random search, and conditional parameter distributions.
- Resource-aware scheduling, including support for concurrent runs of algorithms that may themselves be parallel and distributed.
Concepts
--------
.. image:: images/tune-api.svg
Ray Tune schedules a number of *trials* in a cluster. Each trial runs a user-defined Python function or class and is parameterized by a *config* variation passed to the user code.
In order to run any given function, you need to run ``register_trainable`` to a name. This makes all Ray workers aware of the function.
.. autofunction:: ray.tune.register_trainable
Ray Tune provides a ``run_experiments`` function that generates and runs the trials described by the experiment specification. The trials are scheduled and managed by a *trial scheduler* that implements the search algorithm (default is FIFO).
.. autofunction:: ray.tune.run_experiments
Ray Tune can be used anywhere Ray can, e.g. on your laptop with ``ray.init()`` embedded in a Python script, or in an `auto-scaling cluster <autoscaling.html>`__ for massive parallelism.
You can find the code for Ray Tune `here on GitHub <https://github.com/ray-project/ray/tree/master/python/ray/tune>`__.
Trial Schedulers
----------------
By default, Ray Tune schedules trials in serial order with the ``FIFOScheduler`` class. However, you can also specify a custom scheduling algorithm that can early stop trials, perturb parameters, or incorporate suggestions from an external service. Currently implemented trial schedulers include
`Population Based Training (PBT) <pbt.html>`__, `Median Stopping Rule <hyperband.html#median-stopping-rule>`__, and `HyperBand <hyperband.html>`__.
**Finally**, configure your search and execute it on your Ray cluster:
.. code-block:: python
run_experiments({...}, scheduler=AsyncHyperBandScheduler())
Search Algorithms
-----------------
Tune allows you to use different search algorithms in combination with different scheduling algorithms. Currently, Tune offers the following search algorithms:
- Grid search / Random Search
- Tree-structured Parzen Estimators (HyperOpt)
If you are interested in implementing or contributing a new Search Algorithm, the API is straightforward:
.. autoclass:: ray.tune.suggest.SearchAlgorithm
HyperOpt Integration
~~~~~~~~~~~~~~~~~~~~
The ``HyperOptSearch`` is a SearchAlgorithm that is backed by HyperOpt to perform sequential model-based hyperparameter optimization.
In order to use this search algorithm, you will need to install HyperOpt via the following command:
.. code-block:: bash
$ pip install --upgrade git+git://github.com/hyperopt/hyperopt.git
An example of this can be found in `hyperopt_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py>`__.
.. note::
The HyperOptScheduler takes an *increasing* metric in the reward attribute. If trying to minimize a loss, be sure to
specify *mean_loss* in the function/class reporting and *reward_attr=neg_mean_loss* in the HyperOptScheduler initializer.
.. autoclass:: ray.tune.suggest.HyperOptSearch
Handling Large Datasets
-----------------------
You often will want to compute a large object (e.g., training data, model weights) on the driver and use that object within each trial. Ray Tune provides a ``pin_in_object_store`` utility function that can be used to broadcast such large objects. Objects pinned in this way will never be evicted from the Ray object store while the driver process is running, and can be efficiently retrieved from any task via ``get_pinned_object``.
.. code-block:: python
import ray
from ray.tune import register_trainable, run_experiments
from ray.tune.util import pin_in_object_store, get_pinned_object
import numpy as np
ray.init()
# X_id can be referenced in closures
X_id = pin_in_object_store(np.random.random(size=100000000))
def f(config, reporter):
X = get_pinned_object(X_id)
# use X
register_trainable("f", f)
run_experiments(...)
Visualizing Results
-------------------
Ray Tune logs trial results to a unique directory per experiment, e.g. ``~/ray_results/my_experiment`` in the above example. The log records are compatible with a number of visualization tools:
To visualize learning in tensorboard, install TensorFlow:
.. code-block:: bash
$ pip install tensorflow
Then, after you run a experiment, you can visualize your experiment with TensorBoard by specifying the output directory of your results. Note that if you running Ray on a remote cluster, you can forward the tensorboard port to your local machine through SSH using ``ssh -L 6006:localhost:6006 <address>``:
.. code-block:: bash
$ tensorboard --logdir=~/ray_results/my_experiment
.. image:: ray-tune-tensorboard.png
To use rllab's VisKit (you may have to install some dependencies), run:
.. code-block:: bash
$ git clone https://github.com/rll/rllab.git
$ python rllab/rllab/viskit/frontend.py ~/ray_results/my_experiment
.. image:: ray-tune-viskit.png
Finally, to view the results with a `parallel coordinates visualization <https://en.wikipedia.org/wiki/Parallel_coordinates>`__, open `ParallelCoordinatesVisualization.ipynb <https://github.com/ray-project/ray/blob/master/python/ray/tune/ParallelCoordinatesVisualization.ipynb>`__ as follows and run its cells:
.. code-block:: bash
$ cd $RAY_HOME/python/ray/tune
$ jupyter-notebook ParallelCoordinatesVisualization.ipynb
.. image:: ray-tune-parcoords.png
Trial Checkpointing
-------------------
To enable checkpointing, you must implement a Trainable class (Trainable functions are not checkpointable, since they never return control back to their caller). The easiest way to do this is to subclass the pre-defined ``Trainable`` class and implement its ``_train``, ``_save``, and ``_restore`` abstract methods `(example) <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperband_example.py>`__: Implementing this interface is required to support resource multiplexing in schedulers such as HyperBand and PBT.
For TensorFlow model training, this would look something like this `(full tensorflow example) <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/tune_mnist_ray_hyperband.py>`__:
.. code-block:: python
class MyClass(Trainable):
def _setup(self):
self.saver = tf.train.Saver()
self.sess = ...
self.iteration = 0
def _train(self):
self.sess.run(...)
self.iteration += 1
def _save(self, checkpoint_dir):
return self.saver.save(
self.sess, checkpoint_dir + "/save",
global_step=self.iteration)
def _restore(self, path):
return self.saver.restore(self.sess, path)
Additionally, checkpointing can be used to provide fault-tolerance for experiments. This can be enabled by setting ``checkpoint_freq: N`` and ``max_failures: M`` to checkpoint trials every *N* iterations and recover from up to *M* crashes per trial, e.g.:
.. code-block:: python
run_experiments({
all_trials = tune.run_experiments({
"my_experiment": {
...
"checkpoint_freq": 10,
"max_failures": 5,
},
"run": train_func,
"stop": {"mean_accuracy": 99},
"config": {"momentum": tune.grid_search([0.1, 0.2])}
}
})
The class interface that must be implemented to enable checkpointing is as follows:
Tune can be used anywhere Ray can, e.g. on your laptop with ``ray.init()`` embedded in a Python script, or in an `auto-scaling cluster <autoscaling.html>`__ for massive parallelism.
.. autoclass:: ray.tune.trainable.Trainable
:members: _save, _restore, _train, _setup, _stop
Citing Tune
-----------
If Tune helps you in your academic research, you are encouraged to cite `our paper <https://arxiv.org/abs/1807.05118>`__. Here is an example bibtex:
.. code-block:: tex
@article{liaw2018tune,
title={Tune: A Research Platform for Distributed Model Selection and Training},
author={Liaw, Richard and Liang, Eric and Nishihara, Robert
and Moritz, Philipp and Gonzalez, Joseph E and Stoica, Ion},
journal={arXiv preprint arXiv:1807.05118},
year={2018}
}
Client API
----------
You can modify an ongoing experiment by adding or deleting trials using the Tune Client API. To do this, verify that you have the ``requests`` library installed:
.. code-block:: bash
$ pip install requests
To use the Client API, you can start your experiment with ``with_server=True``:
.. code-block:: python
run_experiments({...}, with_server=True, server_port=4321)
Then, on the client side, you can use the following class. The server address defaults to ``localhost:4321``. If on a cluster, you may want to forward this port (e.g. ``ssh -L <local_port>:localhost:<remote_port> <address>``) so that you can use the Client on your local machine.
.. autoclass:: ray.tune.web_server.TuneClient
:members:
For an example notebook for using the Client API, see the `Client API Example <https://github.com/ray-project/ray/tree/master/python/ray/tune/TuneClient.ipynb>`__.
Examples
--------
You can find a list of examples `using Ray Tune and its various features here <https://github.com/ray-project/ray/tree/master/python/ray/tune/examples>`__, including examples using Keras, TensorFlow, and Population-Based Training.
.. _HyperOpt with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py