[tune] Improve user guides and API docs (#7716)

* create guide gallery for Tune

* mods

* ok

* fix

* fix_up_gallery

* ok

* Apply suggestions from code review

Co-Authored-By: Sven Mika <sven@anyscale.io>

* Apply suggestions from code review

Co-Authored-By: Sven Mika <sven@anyscale.io>

Co-authored-by: Sven Mika <sven@anyscale.io>
This commit is contained in:
Richard Liaw
2020-04-06 12:16:35 -07:00
committed by GitHub
co-authored by Sven Mika
parent 22ccc43670
commit a67edc4051
22 changed files with 574 additions and 340 deletions
+52 -3
View File
@@ -4,6 +4,41 @@ Analysis/Logging (tune.analysis / tune.logger)
Analyzing Results
-----------------
You can use the ``ExperimentAnalysis`` object for analyzing results. It is returned automatically when calling ``tune.run``.
.. code-block:: python
analysis = tune.run(
trainable,
name="example-experiment",
num_samples=10,
)
Here are some example operations for obtaining a summary of your experiment:
.. code-block:: python
# Get a dataframe for the last reported results of all of the trials
df = analysis.dataframe()
# Get a dataframe for the max accuracy seen for each trial
df = analysis.dataframe(metric="mean_accuracy", mode="max")
# Get a dict mapping {trial logdir -> dataframes} for all trials in the experiment.
all_dataframes = analysis.trial_dataframes
# Get a list of trials
trials = analysis.trials
You may want to get a summary of multiple experiments that point to the same ``local_dir``. For this, you can use the ``Analysis`` class.
.. code-block:: python
from ray.tune import Analysis
analysis = Analysis("~/ray_results/example-experiment")
.. _exp-analysis-docstring:
ExperimentAnalysis
~~~~~~~~~~~~~~~~~~
@@ -11,8 +46,6 @@ ExperimentAnalysis
:show-inheritance:
:members:
.. _analysis-docstring:
Analysis
~~~~~~~~
@@ -24,6 +57,21 @@ Analysis
Loggers (tune.logger)
---------------------
Viskit
~~~~~~
Tune automatically integrates with Viskit via the ``CSVLogger`` outputs. To use VisKit (you may have to install some dependencies), run:
.. code-block:: bash
$ git clone https://github.com/rll/rllab.git
$ python rllab/rllab/viskit/frontend.py ~/ray_results/my_experiment
The nonrelevant metrics (like timing stats) can be disabled on the left to show only the relevant ones (like accuracy, loss, etc.).
.. image:: /ray-tune-viskit.png
.. _logger-interface:
Logger
@@ -54,5 +102,6 @@ CSVLogger
MLFLowLogger
~~~~~~~~~~~~
.. autoclass:: ray.tune.logger.MLFLowLogger
Tune also provides a default logger for `MLFlow <https://mlflow.org>`_. You can install MLFlow via ``pip install mlflow``. An example can be found `mlflow_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/mlflow_example.py>`__. Note that this currently does not include artifact logging support. For this, you can use the native MLFlow APIs inside your Trainable definition.
.. autoclass:: ray.tune.logger.MLFLowLogger