[tune] Add Nevergrad to Tune (#3985)

doc/source/tune-searchalg.rst

@@ -9,11 +9,13 @@ You can utilize these search algorithms as follows:
 
     run_experiments(experiments, search_alg=SearchAlgorithm(...))
 
-Currently, Tune offers the following search algorithms:
+Currently, Tune offers the following search algorithms (and library integrations):
 
 - `Grid Search and Random Search <tune-searchalg.html#variant-generation-grid-search-random-search>`__
+- `BayesOpt <tune-searchalg.html#bayesopt-search>`__
 - `HyperOpt <tune-searchalg.html#hyperopt-search-tree-structured-parzen-estimators>`__
 - `SigOpt <tune-searchalg.html#sigopt-search>`__
+- `Nevergrad <tune-searchalg.html#nevergrad-search>`__
 
 
 Variant Generation (Grid Search/Random Search)
@@ -103,6 +105,31 @@ An example of this can be found in `sigopt_example.py <https://github.com/ray-pr
     :show-inheritance:
     :noindex:
 
+Nevergrad Search
+----------------
+
+The ``NevergradSearch`` is a SearchAlgorithm that is backed by `Nevergrad <https://github.com/facebookresearch/nevergrad>`__ to perform sequential model-based hyperparameter optimization. Note that this class does not extend ``ray.tune.suggest.BasicVariantGenerator``, so you will not be able to use Tune's default variant generation/search space declaration when using NevergradSearch.
+
+In order to use this search algorithm, you will need to install Nevergrad via the following command.:
+
+.. code-block:: bash
+
+    $ pip install nevergrad
+
+Keep in mind that ``nevergrad`` is a Python 3.6+ library.
+
+This algorithm requires using an optimizer provided by ``nevergrad``, of which there are many options. A good rundown can be found on their README's `Optimization <https://github.com/facebookresearch/nevergrad>`__ section. You can use ``NevergradSearch`` like follows:
+
+.. code-block:: python
+
+    run_experiments(experiment_config, search_alg=NevergradSearch(optimizer, parameter_names, ... ))
+
+An example of this can be found in `nevergrad_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/nevergrad_example.py>`__.
+
+.. autoclass:: ray.tune.suggest.NevergradSearch
+    :show-inheritance:
+    :noindex:
+
 Scikit-Optimize Search
 ----------------------
 

doc/source/tune.rst

@@ -25,7 +25,7 @@ Features
 
    -  `HyperBand <tune-schedulers.html#asynchronous-hyperband>`__
 
-*  Mix and match different hyperparameter optimization approaches - such as using `HyperOpt with HyperBand`_.
+*  Mix and match different hyperparameter optimization approaches - such as using `HyperOpt with HyperBand`_ or `Nevergrad with HyperBand`_.
 
 *  Visualize results with `TensorBoard <https://www.tensorflow.org/get_started/summaries_and_tensorboard>`__, `parallel coordinates (Plot.ly) <https://plot.ly/python/parallel-coordinates-plot/>`__, and `rllab's VisKit <https://media.readthedocs.org/pdf/rllab/latest/rllab.pdf>`__.
 
@@ -109,3 +109,4 @@ If Tune helps you in your academic research, you are encouraged to cite `our pap
 
 
 .. _HyperOpt with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/hyperopt_example.py
+.. _Nevergrad with HyperBand: https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/nevergrad_example.py