[tune] add more stoppers and stopper documentation (#12750)

* Add new stoppers & docs

* Add tests for maximum iteration stopper and trial plateau stopper

* Update python/ray/tune/stopper.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update doc/source/tune/api_docs/stoppers.rst

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Update doc/source/tune/api_docs/stoppers.rst

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Apply suggestions from code review

* Apply suggestions from code review

* Update python/ray/tune/stopper.py

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

doc/source/tune/api_docs/execution.rst

@@ -23,14 +23,6 @@ tune.with_parameters
 
 .. autofunction:: ray.tune.with_parameters
 
-.. _tune-stop-ref:
-
-Stopper (tune.Stopper)
-----------------------
-
-.. autoclass:: ray.tune.Stopper
-    :members: __call__, stop_all
-
 .. _tune-sync-config:
 
 tune.SyncConfig

doc/source/tune/api_docs/overview.rst

@@ -21,6 +21,7 @@ on `Github`_.
    suggestion.rst
    schedulers.rst
    sklearn.rst
+   stoppers.rst
    logging.rst
    integration.rst
    internals.rst

doc/source/tune/api_docs/stoppers.rst

@@ -0,0 +1,46 @@
+.. _tune-stoppers:
+
+Stopping mechanisms (tune.stopper)
+==================================
+
+In addition to Trial Schedulers like :ref:`ASHA <tune-scheduler-hyperband>`, where a number of
+trials are stopped if they perform subpar, Ray Tune also supports custom stopping mechanisms to stop trials early. For instance, stopping mechanisms can specify to stop trials when they reached a plateau and the metric
+doesn't change anymore.
+
+Ray Tune comes with several stopping mechanisms out of the box. For custom stopping behavior, you can
+inherit from the :class:`Stopper <ray.tune.Stopper>` class.
+
+Other stopping behaviors are described :ref:`in the user guide <tune-stopping>`.
+
+.. contents::
+    :local:
+    :depth: 1
+
+
+.. _tune-stop-ref:
+
+Stopper (tune.Stopper)
+----------------------
+
+.. autoclass:: ray.tune.Stopper
+    :members: __call__, stop_all
+
+MaximumIterationStopper (tune.stopper.MaximumIterationStopper)
+--------------------------------------------------------------
+
+.. autoclass:: ray.tune.stopper.MaximumIterationStopper
+
+ExperimentPlateauStopper (tune.stopper.ExperimentPlateauStopper)
+----------------------------------------------------------------
+
+.. autoclass:: ray.tune.stopper.ExperimentPlateauStopper
+
+TrialPlateauStopper (tune.stopper.TrialPlateauStopper)
+------------------------------------------------------
+
+.. autoclass:: ray.tune.stopper.TrialPlateauStopper
+
+TimeoutStopper (tune.stopper.TimeoutStopper)
+--------------------------------------------
+
+.. autoclass:: ray.tune.stopper.TimeoutStopper

doc/source/tune/user-guide.rst

@@ -305,7 +305,9 @@ and passed to your trainable as a parameter.
 Stopping Trials
 ---------------
 
-You can control when trials are stopped early by passing the ``stop`` argument to ``tune.run``. This argument takes either a dictionary or a function.
+You can control when trials are stopped early by passing the ``stop`` argument to ``tune.run``.
+This argument takes, a dictionary, a function, or a :class:`Stopper <ray.tune.stopper.Stopper>` class
+as an argument.
 
 If a dictionary is passed in, the keys may be any field in the return result of ``tune.report`` in the Function API or ``step()`` (including the results from ``step`` and auto-filled metrics).
 
@@ -329,7 +331,7 @@ For more flexibility, you can pass in a function instead. If a function is passe
 
     tune.run(my_trainable, stop=stopper)
 
-Finally, you can implement the ``Stopper`` abstract class for stopping entire experiments. For example, the following example stops all trials after the criteria is fulfilled by any individual trial, and prevents new ones from starting:
+Finally, you can implement the :class:`Stopper <ray.tune.stopper.Stopper>` abstract class for stopping entire experiments. For example, the following example stops all trials after the criteria is fulfilled by any individual trial, and prevents new ones from starting:
 
 .. code-block:: python
 
@@ -352,7 +354,9 @@ Finally, you can implement the ``Stopper`` abstract class for stopping entire ex
     tune.run(my_trainable, stop=stopper)
 
 
-Note that in the above example the currently running trials will not stop immediately but will do so once their current iterations are complete. See the :ref:`tune-stop-ref` documentation.
+Note that in the above example the currently running trials will not stop immediately but will do so once their current iterations are complete.
+
+Ray Tune comes with a set of out-of-the-box stopper classes. See the :ref:`Stopper <tune-stoppers>` documentation.
 
 .. _tune-logging: