wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-12 12:20:11 +08:00
Code Issues Packages Projects Releases Wiki Activity

[tune] Adds option to checkpoint at end of trials (#2754)

Browse Source 
* Added checkpoint_at_end option. To fix #2740

* Added ability to checkpoint at the end of trials if the option is set to True

* checkpoint_at_end option added; Consistent with Experience and Trial runner

* checkpoint_at_end option mentioned in the tune usage guide

* Moved the redundant checkpoint criteria check out of the if-elif

* Added note that checkpoint_at_end is enabled only when checkpoint_freq is not 0

* Added test case for checkpoint_at_end

* Made checkpoint_at_end have an effect regardless of checkpoint_freq

* Removed comment from the test case

* Fixed the indentation

* Fixed pep8 E231

* Handled cases when trainable does not have _save implemented

* Constrained test case to a particular exp using the MockAgent

* Revert "Constrained test case to a particular exp using the MockAgent"

This reverts commit e965a9358ec7859b99a3aabb681286d6ba3c3906.

* Revert "Handled cases when trainable does not have _save implemented"

This reverts commit 0f5382f996ff0cbf3d054742db866c33494d173a.

* Simpler test case for checkpoint_at_end

* Preserved bools from loosing their actual value

* Revert "Moved the redundant checkpoint criteria check out of the if-elif"

This reverts commit 783005122902240b0ee177e9e206e397356af9c5.

* Fix linting error.
This commit is contained in:
Praveen Palanisamy
2018-08-29 13:14:17 -07:00
committed by Richard Liaw
parent 6edbbf4fbf
commit 357c0d6156
6 changed files with 59 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/source/tune-usage.rst
+14
View File
@@ -254,6 +254,20 @@ Additionally, checkpointing can be used to provide fault-tolerance for experimen
},
})
The checkpoint_freq may not coincide with the exact end of an experiment. If you want a checkpoint to be created at the end
of a trial, you can additionally set the checkpoint_at_end to True. An example is shown below:
.. code-block:: python
:emphasize-lines: 5
run_experiments({
"my_experiment_name": {
"run": my_trainable
"checkpoint_freq": 10,
"checkpoint_at_end": True,
"max_failures": 5,
},
})
Handling Large Datasets
-----------------------