wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-02 13:01:01 +08:00
Code Issues Packages Projects Releases Wiki Activity

[tune] Adds option to checkpoint at end of trials (#2754)

Browse Source 
* Added checkpoint_at_end option. To fix #2740

* Added ability to checkpoint at the end of trials if the option is set to True

* checkpoint_at_end option added; Consistent with Experience and Trial runner

* checkpoint_at_end option mentioned in the tune usage guide

* Moved the redundant checkpoint criteria check out of the if-elif

* Added note that checkpoint_at_end is enabled only when checkpoint_freq is not 0

* Added test case for checkpoint_at_end

* Made checkpoint_at_end have an effect regardless of checkpoint_freq

* Removed comment from the test case

* Fixed the indentation

* Fixed pep8 E231

* Handled cases when trainable does not have _save implemented

* Constrained test case to a particular exp using the MockAgent

* Revert "Constrained test case to a particular exp using the MockAgent"

This reverts commit e965a9358ec7859b99a3aabb681286d6ba3c3906.

* Revert "Handled cases when trainable does not have _save implemented"

This reverts commit 0f5382f996ff0cbf3d054742db866c33494d173a.

* Simpler test case for checkpoint_at_end

* Preserved bools from loosing their actual value

* Revert "Moved the redundant checkpoint criteria check out of the if-elif"

This reverts commit 783005122902240b0ee177e9e206e397356af9c5.

* Fix linting error.
This commit is contained in:
Praveen Palanisamy
2018-08-29 13:14:17 -07:00
committed by Richard Liaw
parent 6edbbf4fbf
commit 357c0d6156
6 changed files with 59 additions and 2 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/ray/tune/trial.py
+6 -1
View File
@@ -112,6 +112,7 @@ class Trial(object):
resources=None,
stopping_criterion=None,
checkpoint_freq=0,
checkpoint_at_end=False,
restore_path=None,
upload_dir=None,
max_failures=0):
@@ -142,6 +143,7 @@ class Trial(object):
# Local trial state that is updated during the run
self.last_result = None
self.checkpoint_freq = checkpoint_freq
self.checkpoint_at_end = checkpoint_at_end
self._checkpoint = Checkpoint(
storage=Checkpoint.DISK, value=restore_path)
self.status = Trial.PENDING
@@ -203,9 +205,12 @@ class Trial(object):
return False
def should_checkpoint(self):
def should_checkpoint(self, result):
"""Whether this trial is due for checkpointing."""
if result.get(DONE) and self.checkpoint_at_end:
return True
if not self.checkpoint_freq:
return False