Commit Graph
87 Commits
Author SHA1 Message Date
Kai FrickeandGitHub c77cfaa5ad [tune] use dated experiment dir per default (#11104) 2020-09-30 14:43:59 -07:00
Kai FrickeandGitHub d9c4dea7cf [tune] strict metric checking (#10972) 2020-09-24 10:00:48 -07:00
Kai FrickeandGitHub ee99c919e3 [tune] lazy trials (#10802)
* Lazily fill trial queue

* Update interface

* Update end to end reporter test

* Removed `next_trials()` method

* Lint

* Print total number of samples to be generated in progress reporter. Allow infinite samples.

* Nit check
2020-09-17 08:51:46 -07:00
c9fafe7733 [tune] added type hints (#10806)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-15 21:03:56 -07:00
3501ea396c [tune] All examples to use ConcurrencyLimiter (#10662)
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-09-09 09:52:15 -07:00
d7c7aba99c [tune] Tune experiment analysis improvements (#10645)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-08 21:00:52 -07:00
756a9ea641 [tune] add mode/metric parameters to tune.run (#10627)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-08 17:06:21 -07:00
Kai FrickeandGitHub 088f8ebb69 [tune] Add algorithms for search space conversion (#10621) 2020-09-07 13:44:16 -07:00
54215ff287 [tune] implement shim instantiation (#10456)
* Create ray.tune.suggest.create.create_scheduler

* Update __init__.py

* Resolve conflict in __init__.py

* Create ray.tune.schedulers.create.create_scheduler

* Update __init__.py

* Move create_scheduler to tune.schedulers.__init__

* Move create_searcher to tune.suggest.__init__

* Delete tune.suggest.create

* Delete tune.schedulers.create

* Update imports for shim functions in tune.__init__

* Remove shim from tune.suggest.__init__.__all__

* Remove shim from tune.schedulers.__init__.__all__

* Add ShimCreationTest

* Move ShimCreationTest to test_api

* Delete test_shim.py

* Add docstring for ray.tune.create_scheduler

* Add docstring to ray.tune.create_searcher

* Fix typo in ray.tune.create_scheduler docstring

* Fix lint errors in tune.schedulers.__init__

* Fix lint errors in tune.suggest.__init__

* Fix lint errors in tune.suggest.__init__

* Fix lint errors in tune.schedulers.__init__

* Fix imports in test_api

* Fix lint errors in test_api

* Fix kwargs in create_searcher

* Fix kwargs in create_scheduler

* Merge branch 'master' into shim-instantiation

* Update use-case in docs in tune.create_scheduler

* Update use-case in docs in tune.create_searcher

* Remove duplicate pytest run from test_api

* Add check to create_searcher


Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Add check to create_scheduler

* lint

* Compare types of instances in test_api

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>

* Add tune.create_searcher to docs

* Fix doc build

* Fix tests

* Add tune.create_scheduler to docs

* Fix tests

* Fix lint errors

* Update Ax search for master

* Fix metric kwarg for Ax in test_api

* Fix doc build

* Fix HyperOptSearch import in test_api

* Fix HyperOptSearch import in create_searcher

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-05 09:36:42 -07:00
2fac66650d [tune] extend search space api docs (#10576)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-04 18:39:51 -07:00
06af62ba91 [tune] refactor tune search space (#10444)
* Added basic functionality and tests

* Feature parity with old tune search space config

* Convert Optuna search spaces

* Introduced quantized values

* Updated Optuna resolving

* Added HyperOpt search space conversion

* Convert search spaces to AxSearch

* Convert search spaces to BayesOpt

* Added basic functionality and tests

* Feature parity with old tune search space config

* Convert Optuna search spaces

* Introduced quantized values

* Updated Optuna resolving

* Added HyperOpt search space conversion

* Convert search spaces to AxSearch

* Convert search spaces to BayesOpt

* Re-factored samplers into domain classes

* Re-added base classes

* Re-factored into list comprehensions

* Added `from_config` classmethod for config conversion

* Applied suggestions from code review

* Removed truncated normal distribution

* Set search properties in tune.run

* Added test for tune.run search properties

* Move sampler initializers to base classes

* Add tune API sampling test, fixed includes, fixed resampling bug

* Add to API docs

* Fix docs

* Update metric and mode only when set. Set default metric and mode to experiment analysis object.

* Fix experiment analysis tests

* Raise error when delimiter is used in the config keys

* Added randint/qrandint to API docs, added additional check in tune.run

* Fix tests

* Fix linting error

* Applied suggestions from code review. Re-aded tune.function for the time being

* Fix sampling tests

* Fix experiment analysis tests

* Fix tests and linting error

* Removed unnecessary default_config attribute from OptunaSearch

* Revert to set AxSearch default metric

* fix-min-max

* fix

* nits

* Added function check, enhanced loguniform error message

* fix-print

* fix

* fix

* Raise if unresolved values are in config and search space is already set

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-09-03 09:06:13 -07:00
raoul-khour-tsandGitHub 3b10b67a15 [tune] SigOpt multi-objective search + experiments (#10457) 2020-09-01 16:22:29 -07:00
Richard LiawandGitHub 09d4a3241f [tune] Support true pooling and batched concurrency (#10352) 2020-09-01 10:33:49 -07:00
Sumanth RatnaandGitHub bac8f14739 [tune] Use isinstance instead of type in assert in HyperOptSearch (#10454) 2020-08-31 15:26:55 -07:00
raoul-khour-tsandGitHub 25f5614691 [tune] Rrk/sigopt_searcher_improvements (#10446) 2020-08-31 13:15:12 -07:00
87ed20738e [tune] Add on_pause, on_unpause to ConcurrencyLimiter (#10320)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-08-25 22:33:17 -07:00
Richard LiawandGitHub 146d91385c [tune] custom trial directory name (#10214) 2020-08-25 12:52:54 -07:00
16486a8df3 [tune] Add OptunaSearcher wrapper around Optuna samplers (#10044)
Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
Co-authored-by: Kai Fricke <kai@anyscale.com>
2020-08-12 16:13:22 -07:00
Richard LiawandGitHub 7a8b922841 [tune] hotfix log_once (#10069) 2020-08-12 12:40:22 -07:00
Richard LiawandGitHub be8e63d477 [tune] support resume for search algorithms (#9972) 2020-08-10 13:43:14 -07:00
c6404e8cf6 [tune] Search alg checkpointing during training (#9803)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-08-03 15:07:31 -07:00
6c49c01837 [tune] Function API checkpointing (#8471)
Co-authored-by: krfricke <krfricke@users.noreply.github.com>
2020-06-15 10:42:54 -07:00
Jack CarreiraandGitHub 19cc1ae781 [docs] Tune Search: Wrong parameter name (#8927) 2020-06-13 18:01:22 -07:00
Sumanth RatnaandGitHub 57212254e6 Fix dragonfly install instructions (#8866) 2020-06-09 13:05:04 -07:00
Richard LiawandGitHub d7b64ef279 [tune] BayesOpt - finish early when optimizer converges (#8808) 2020-06-09 12:09:39 -07:00
Luca CappellettiandGitHub 822de1b7f7 [Tune] Introduced preliminary random search to BayesOpt (#8541) 2020-05-23 12:20:43 -07:00
Luca CappellettiandGitHub c9898eff24 [Tune] Added method to integrate previous analysis in BO (#8486) 2020-05-19 23:26:43 -07:00
Luca CappellettiandGitHub d1ef70da16 [Tune] Added default values for utility kwargs (#8488) 2020-05-18 13:10:43 -07:00
Richard LiawandGitHub 87cbf2aedd [docs][tune] Make search algorithm, scheduler docs better! (#8179) 2020-05-17 12:19:44 -07:00
Eric LiangandGitHub 1126fe4d23 [tune] Add UUID back to trial names (#8377) 2020-05-08 20:20:36 -07:00
Richard LiawandGitHub 40dfb337bf [tune] Hotfix Ax breakage when fixing backwards-compat (#8285) 2020-05-02 20:42:50 -07:00
Richard LiawandGitHub 07daff8794 [tune] Avoid breakage - soft deprecation warning for search algs (#8258) 2020-05-01 10:36:43 -07:00
87557a00fa [tune] Refactor search algorithms (#7037)
* start refactoring of search algorithms

* format

* needs tests

* fix

* suggestions

* Fix PBT

* lint

* refactoring

* hyperopt_working

* dragonfly

* hyperopt

* change_half_of_algs

* save

* code-removed

* remove_lots_of_unneccessary

* changes

* formatting

* suggest

* reset

* rm

* tests

* search-change

* exception

* refactor-doc

* search

* py

* moredocs

* Update doc/source/tune-searchalg.rst

* concurrency

* max

* tune

* betterwarning

* bohb

* tests

* test-change

Co-authored-by: ujvl <misraujval@gmail.com>
2020-04-27 08:51:13 -07:00
Richard LiawandGitHub 4d8bf5635d [hotfix] Lint formatting for new Tune optimizer ZOOpt (#8040)
* formatting

* removedill

* lint
2020-04-16 09:24:30 -07:00
5c274fe631 [Tune] Add ZOOpt search algorithm (#7960)
* add zoopt

* add zoopt search algo

* add zoopt

* fix zoopt

* add zoopt requirements

* fix zoopt

* remove generated guides

* Apply suggestions from code review

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-04-15 21:13:29 -07:00
54a892bb84 [tune] Cancel Experiment via Client (#7719)
* init cancel

* testing

* Update python/ray/tune/tests/test_tune_server.py

Co-Authored-By: Richard Liaw <rliaw@berkeley.edu>

* Apply suggestions from code review

* Apply suggestions from code review

* finished

* set_finished

Co-authored-by: ijrsvt <ian.rodney@gmail.com>
2020-03-24 20:30:12 -07:00
81d311031b [tune] Update API Reference Page (#7671)
* widerdocs

* init

* docs

* fix

* moveit

* mix

* better_docs

* remove

* Apply suggestions from code review

Co-Authored-By: Sven Mika <sven@anyscale.io>

Co-authored-by: Sven Mika <sven@anyscale.io>
2020-03-22 16:42:20 -07:00
094125cf03 [tune] Dragonfly integration ask tell nit (#7593)
* Add sample example

* Copy relevant lines of ask from inherited Optimizer

* Ignore strategy

* Additional changes

* Add DragonflySearch for tune connector for Dragonfly

* Add example and fix small errors

* lint

* Remove skopt references

* Update example based off of Dragonfly changes

* Edit example for final Dragonfly edits

* Formatting and documentation edits

* Add documentation and add to test pipeline

* Address PR comments

* Fix Jenkins test

* Adjust Dragonfly to PR#7366

* Lint

* fix_tests

* Minor changes to ordering

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-03-13 15:27:03 -07:00
89ec4adb72 [tune] Dragonfly Optimizer (#5955)
* Add sample example

* Copy relevant lines of ask from inherited Optimizer

* Ignore strategy

* Additional changes

* Add DragonflySearch for tune connector for Dragonfly

* Add example and fix small errors

* lint

* Remove skopt references

* Update example based off of Dragonfly changes

* Edit example for final Dragonfly edits

* Formatting and documentation edits

* Add documentation and add to test pipeline

* Address PR comments

* Fix Jenkins test

* Adjust Dragonfly to PR#7366

* Lint

* fix_tests

Co-authored-by: Richard Liaw <rliaw@berkeley.edu>
2020-03-10 08:40:36 -07:00
115468de2c [tune] Repeated evals (#7366)
* easyrepeat

* done

* suggest

* doc

* ok

* commit

* Apply suggestions from code review

Co-Authored-By: Ujval Misra <misraujval@gmail.com>

* Apply suggestions from code review

Co-Authored-By: Ujval Misra <misraujval@gmail.com>

* Apply suggestions from code review

* ok

* docs

Co-authored-by: Ujval Misra <misraujval@gmail.com>
2020-03-07 11:08:23 -08:00
Eric LiangandGitHub 1ea05a2c08 [tune] Fix a number of reporter regressions and add end-to-end tests (#7274) 2020-02-25 14:31:56 -08:00
Richard LiawandGitHub 4edfaf2f38 [tune] Support callable objects in variant generation (#6849)
* minorcallable

* format
2020-01-21 10:24:25 -08:00
SvenandEric Liang 60d4d5e1aa Remove future imports (#6724)
* Remove all __future__ imports from RLlib.

* Remove (object) again from tf_run_builder.py::TFRunBuilder.

* Fix 2xLINT warnings.

* Fix broken appo_policy import (must be appo_tf_policy)

* Remove future imports from all other ray files (not just RLlib).

* Remove future imports from all other ray files (not just RLlib).

* Remove future import blocks that contain `unicode_literals` as well.
Revert appo_tf_policy.py to appo_policy.py (belongs to another PR).

* Add two empty lines before Schedule class.

* Put back __future__ imports into determine_tests_to_run.py. Fails otherwise on a py2/print related error.
2020-01-09 00:15:48 -08:00
Ujval MisraandRichard Liaw 20ba7ef647 [tune] Move util to utils package (#6682)
* Move util.py to utils

* Fix import
2020-01-06 18:11:02 -08:00
Robert NishiharaandPhilipp Moritz 39a3459886 Remove (object) from class declarations. (#6658) 2020-01-02 17:42:13 -08:00
Adi ZimmermanandRichard Liaw 776b071f3b [tune] Let Search Algorithms use early stopped trials (#5651) 2019-11-11 09:38:14 -08:00
Hersh GodseandRichard Liaw d17b35494d [tune] Save/Restore for Suggestion Algs (#5719) 2019-09-21 11:11:57 -07:00
Vince JankovicsandRichard Liaw 7e214fd95e [tune] TensorBoard HParams for TF2.0 (#5678) 2019-09-21 11:06:34 -07:00
Hersh GodseandRichard Liaw 336aef1774 [tune] Save and Restore for bayesopt (#5623) 2019-09-10 13:11:59 -07:00
Richard LiawandEric Liang 34f6d2fc5c [tune] Update trainable docs and support hparams (#5558) 2019-09-04 12:44:42 -07:00