[tune] Dragonfly Optimizer (#5955)

* Add sample example

* Copy relevant lines of ask from inherited Optimizer

* Ignore strategy

* Additional changes

* Add DragonflySearch for tune connector for Dragonfly

* Add example and fix small errors

* lint

* Remove skopt references

* Update example based off of Dragonfly changes

* Edit example for final Dragonfly edits

* Formatting and documentation edits

* Add documentation and add to test pipeline

* Address PR comments

* Fix Jenkins test

* Adjust Dragonfly to PR#7366

* Lint

* fix_tests

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

doc/source/tune-searchalg.rst

@@ -172,7 +172,7 @@ In order to use this search algorithm, you will need to install Scikit-Optimize
 
     $ pip install scikit-optimize
 
-This algorithm requires using the `Scikit-Optimize ask and tell interface <https://scikit-optimize.github.io/notebooks/ask-and-tell.html>`__. This interface requires using the `Optimizer <https://scikit-optimize.github.io/#skopt.Optimizer>`__ provided by Scikit-Optimize. You can use SkOptSearch like follows:
+This algorithm requires using the `Scikit-Optimize ask and tell interface <https://scikit-optimize.github.io/notebooks/ask-and-tell.html>`__. This interface requires using the `Optimizer <https://scikit-optimize.github.io/#skopt.Optimizer>`__ provided by Scikit-Optimize. You can use SkOptSearch like follows: 
 
 .. code-block:: python
 
@@ -185,6 +185,33 @@ An example of this can be found in `skopt_example.py <https://github.com/ray-pro
     :show-inheritance:
     :noindex:
 
+Dragonfly Search
+----------------
+
+The ``DragonflySearch`` is a SearchAlgorithm that is backed by `Dragonfly <https://github.com/dragonfly/dragonfly>`__ to perform sequential Bayesian optimization. Note that this class does not extend ``ray.tune.suggest.BasicVariantGenerator``, so you will not be able to use Tune's default variant generation/search space declaration when using DragonflySearch.
+
+.. code-block:: bash
+
+    $ pip install dragonfly
+
+This algorithm requires using the `Dragonfly ask and tell interface <https://dragonfly-opt.readthedocs.io/en/master/getting_started_ask_tell/>`__. This interface requires using FunctionCallers and optimizers provided by Dragonfly. You can use `DragonflySearch` like follows:
+
+.. code-block:: python
+
+    from dragonfly.opt.gp_bandit import EuclideanGPBandit
+    from dragonfly.exd.experiment_caller import EuclideanFunctionCaller
+    from dragonfly import load_config
+    domain_config = load_config({'domain': ...})
+    func_caller = EuclideanFunctionCaller(None, domain_config.domain.list_of_domains[0])
+    optimizer = EuclideanGPBandit(func_caller, ask_tell_mode=True)
+    algo = DragonflySearch(optimizer, ...)
+
+An example of this can be found in `dragonfly_example.py <https://github.com/ray-project/ray/blob/master/python/ray/tune/examples/dragonfly_example.py>`__.
+
+.. autoclass:: ray.tune.suggest.dragonfly.DragonflySearch
+    :show-inheritance:
+    :noindex:
+
 Ax Search
 ---------