## What do these changes do?
Provides a small helper function for modifying the resource requirements of a trial.
Also implements the following:
- setting the last_result to be {} instead of None
- Adding a shuffle to the BasicVariantGenerator
In earlier PRs, PR#3585 and PR#3637, export_policy_model and export_policy_checkpoint were introduced for users to export TensorFlow model and checkpoint.
For Ray Tune users, these APIs are not accessible through YAML configurations.
In this pull request, export_formats option is provided to enable users to choose the desired export format.
This PR introduces cluster-level fault tolerance for Tune by checkpointing global state. This occurs with relatively high frequency and allows users to easily resume experiments when the cluster crashes.
Note that this PR may affect automated workflows due to auto-prompting, but this is resolvable.
This PR introduces single-node fault tolerance for Tune.
## Previous behavior:
- Actors will be restarted without checking if resources are available. This can lead to problems if we lose resources.
## New behavior:
- RUNNING trials will be resumed on another node on a best effort basis (meaning they will run if resources available).
- If the cluster is saturated, RUNNING trials on that failed node will become PENDING and queued.
- During recovery, TrialSchedulers and SearchAlgorithms should receive notification of this (via `trial_runner.stop_trial`) so that they don’t wait/block for a trial that isn’t running.
Remaining questions:
- Should `last_result` be consistent during restore?
Yes; but not for earlier trials (trials that are yet to be checkpointed).
- Waiting for some PRs to merge first (#3239)
Closes#2851.
* Added checkpoint_at_end option. To fix#2740
* Added ability to checkpoint at the end of trials if the option is set to True
* checkpoint_at_end option added; Consistent with Experience and Trial runner
* checkpoint_at_end option mentioned in the tune usage guide
* Moved the redundant checkpoint criteria check out of the if-elif
* Added note that checkpoint_at_end is enabled only when checkpoint_freq is not 0
* Added test case for checkpoint_at_end
* Made checkpoint_at_end have an effect regardless of checkpoint_freq
* Removed comment from the test case
* Fixed the indentation
* Fixed pep8 E231
* Handled cases when trainable does not have _save implemented
* Constrained test case to a particular exp using the MockAgent
* Revert "Constrained test case to a particular exp using the MockAgent"
This reverts commit e965a9358ec7859b99a3aabb681286d6ba3c3906.
* Revert "Handled cases when trainable does not have _save implemented"
This reverts commit 0f5382f996ff0cbf3d054742db866c33494d173a.
* Simpler test case for checkpoint_at_end
* Preserved bools from loosing their actual value
* Revert "Moved the redundant checkpoint criteria check out of the if-elif"
This reverts commit 783005122902240b0ee177e9e206e397356af9c5.
* Fix linting error.
A bunch of minor rllib fixes:
pull in latest baselines atari wrapper changes (and use deepmind wrapper by default)
move reward clipping to policy evaluator
add a2c variant of a3c
reduce vision network fc layer size to 256 units
switch to 84x84 images
doc tweaks
print timesteps in tune status
This PR introduces the following changes:
* Ray Tune -> Tune
* [breaking] Creation of `schedulers/`, moving PBT, HyperBand into a submodule
* [breaking] Search Algorithms now must take in experiment configurations via `add_configurations` rather through initialization
* Support `"run": (function | class | str)` with automatic registering of trainable
* Documentation Changes
* Make ActorHandles pickleable, also make proper ActorHandle and ActorClass classes.
* Fix bug.
* Fix actor test bug.
* Update __ray_terminate__ usage.
* Fix most linting, add documentation, and small cleanups.
* Handle forking and pickling differently for actor handles. Fix linting.
* Fixes for named actors via pickling.
* Generate actor handle IDs deterministically in the pickling case.
* Fix the case where Trial logs into wrong paths when `local_dir`
argument starts with tilde (~), by expanding the `local_dir` argument
* Add test case for checking that the tilde gets expanded
* Treat actor creation like a regular task.
* Small cleanups.
* Change semantics of actor resource handling.
* Bug fix.
* Minor linting
* Bug fix
* Fix jenkins test.
* Fix actor tests
* Some cleanups
* Bug fix
* Fix bug.
* Remove cached actor tasks when a driver is removed.
* Add more info to taskspec in global state API.
* Fix cyclic import bug in tune.
* Fix
* Fix linting.
* Fix linting.
* Don't schedule any tasks (especially actor creaiton tasks) on local schedulers with 0 CPUs.
* Bug fix.
* Add test for 0 CPU case
* Fix linting
* Address comments.
* Fix typos and add comment.
* Add assertion and fix test.
* patch up pbt
* Sat Jan 27 01:00:03 PST 2018
* Sat Jan 27 01:04:14 PST 2018
* Sat Jan 27 01:04:21 PST 2018
* Sat Jan 27 01:15:15 PST 2018
* Sat Jan 27 01:15:42 PST 2018
* Sat Jan 27 01:16:14 PST 2018
* Sat Jan 27 01:38:42 PST 2018
* Sat Jan 27 01:39:21 PST 2018
* add pbt
* Sat Jan 27 01:41:19 PST 2018
* Sat Jan 27 01:44:21 PST 2018
* Sat Jan 27 01:45:46 PST 2018
* Sat Jan 27 16:54:42 PST 2018
* Sat Jan 27 16:57:53 PST 2018
* clean up test
* Sat Jan 27 18:01:15 PST 2018
* Sat Jan 27 18:02:54 PST 2018
* Sat Jan 27 18:11:18 PST 2018
* Sat Jan 27 18:11:55 PST 2018
* Sat Jan 27 18:14:09 PST 2018
* review
* try out a ppo example
* some tweaks to ppo example
* add postprocess hook
* Sun Jan 28 15:00:40 PST 2018
* clean up custom explore fn
* Sun Jan 28 15:10:21 PST 2018
* Sun Jan 28 15:14:53 PST 2018
* Sun Jan 28 15:17:04 PST 2018
* Sun Jan 28 15:33:13 PST 2018
* Sun Jan 28 15:56:40 PST 2018
* Sun Jan 28 15:57:36 PST 2018
* Sun Jan 28 16:00:35 PST 2018
* Sun Jan 28 16:02:58 PST 2018
* Sun Jan 28 16:29:50 PST 2018
* Sun Jan 28 16:30:36 PST 2018
* Sun Jan 28 16:31:44 PST 2018
* improve tune doc
* concepts
* update humanoid
* Fri Feb 2 18:03:33 PST 2018
* fix example
* show error file
Remove rllib dep: trainable is now a standalone abstract class that can be easily subclassed.
Clean up hyperband: fix debug string and add an example.
Remove YAML api / ScriptRunner: this was never really used.
Move ray.init() out of run_experiments(): This provides greater flexibility and should be less confusing since there isn't an implicit init() done there. Note that this is a breaking API change for tune.
* wip
* Sat Dec 30 15:07:28 PST 2017
* log video
* video doesn't work well
* scenario integration
* Sat Dec 30 17:30:22 PST 2017
* Sat Dec 30 17:31:05 PST 2017
* Sat Dec 30 17:31:32 PST 2017
* Sat Dec 30 17:32:16 PST 2017
* Sat Dec 30 17:34:11 PST 2017
* Sat Dec 30 17:34:50 PST 2017
* Sat Dec 30 17:35:34 PST 2017
* Sat Dec 30 17:38:49 PST 2017
* Sat Dec 30 17:40:39 PST 2017
* Sat Dec 30 17:43:00 PST 2017
* Sat Dec 30 17:43:04 PST 2017
* Sat Dec 30 17:45:56 PST 2017
* Sat Dec 30 17:46:26 PST 2017
* Sat Dec 30 17:47:02 PST 2017
* Sat Dec 30 17:51:53 PST 2017
* Sat Dec 30 17:52:54 PST 2017
* Sat Dec 30 17:56:43 PST 2017
* Sat Dec 30 18:27:07 PST 2017
* Sat Dec 30 18:27:52 PST 2017
* fix train
* Sat Dec 30 18:41:51 PST 2017
* Sat Dec 30 18:54:11 PST 2017
* Sat Dec 30 18:56:22 PST 2017
* Sat Dec 30 19:05:04 PST 2017
* Sat Dec 30 19:05:23 PST 2017
* Sat Dec 30 19:11:53 PST 2017
* Sat Dec 30 19:14:31 PST 2017
* Sat Dec 30 19:16:20 PST 2017
* Sat Dec 30 19:18:05 PST 2017
* Sat Dec 30 19:18:45 PST 2017
* Sat Dec 30 19:22:44 PST 2017
* Sat Dec 30 19:24:41 PST 2017
* Sat Dec 30 19:26:57 PST 2017
* Sat Dec 30 19:40:37 PST 2017
* wip models
* reward bonus
* test prep
* Sun Dec 31 18:45:25 PST 2017
* Sun Dec 31 18:58:28 PST 2017
* Sun Dec 31 18:59:34 PST 2017
* Sun Dec 31 19:03:33 PST 2017
* Sun Dec 31 19:05:05 PST 2017
* Sun Dec 31 19:09:25 PST 2017
* fix train
* kill
* add tuple preprocessor
* Sun Dec 31 20:38:33 PST 2017
* Sun Dec 31 22:51:24 PST 2017
* Sun Dec 31 23:14:13 PST 2017
* Sun Dec 31 23:16:04 PST 2017
* Mon Jan 1 00:08:35 PST 2018
* Mon Jan 1 00:10:48 PST 2018
* Mon Jan 1 01:08:31 PST 2018
* Mon Jan 1 14:45:44 PST 2018
* Mon Jan 1 14:54:56 PST 2018
* Mon Jan 1 17:29:29 PST 2018
* switch to euclidean dists
* Mon Jan 1 17:39:27 PST 2018
* Mon Jan 1 17:41:47 PST 2018
* Mon Jan 1 17:44:18 PST 2018
* Mon Jan 1 17:47:09 PST 2018
* Mon Jan 1 20:31:02 PST 2018
* Mon Jan 1 20:39:33 PST 2018
* Mon Jan 1 20:40:55 PST 2018
* Mon Jan 1 20:55:06 PST 2018
* Mon Jan 1 21:05:52 PST 2018
* fix env path
* merge richards fix
* fix hash
* Mon Jan 1 22:04:00 PST 2018
* Mon Jan 1 22:25:29 PST 2018
* Mon Jan 1 22:30:42 PST 2018
* simplified reward function
* add framestack
* add env configs
* simplify speed reward
* Tue Jan 2 17:36:15 PST 2018
* Tue Jan 2 17:49:16 PST 2018
* Tue Jan 2 18:10:38 PST 2018
* add lane keeping simple mode
* Tue Jan 2 20:25:26 PST 2018
* Tue Jan 2 20:30:30 PST 2018
* Tue Jan 2 20:33:26 PST 2018
* Tue Jan 2 20:41:42 PST 2018
* ppo lane keep
* simplify discrete actions
* Tue Jan 2 21:41:05 PST 2018
* Tue Jan 2 21:49:03 PST 2018
* Tue Jan 2 22:12:23 PST 2018
* Tue Jan 2 22:14:42 PST 2018
* Tue Jan 2 22:20:59 PST 2018
* Tue Jan 2 22:23:43 PST 2018
* Tue Jan 2 22:26:27 PST 2018
* Tue Jan 2 22:27:20 PST 2018
* Tue Jan 2 22:44:00 PST 2018
* Tue Jan 2 22:57:58 PST 2018
* Tue Jan 2 23:08:51 PST 2018
* Tue Jan 2 23:11:32 PST 2018
* update dqn reward
* Thu Jan 4 12:29:40 PST 2018
* Thu Jan 4 12:30:26 PST 2018
* Update train_dqn.py
* fix
