wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-06-27 22:08:16 +08:00
Code Issues Packages Projects Releases Wiki Activity

[tune] Fix worker recovery by setting force=False when calling logger sync_now (#4302)

Browse Source 
## What do these changes do?
Fixes a tune autoscaling problem where worker recovery causes things to stall.
This commit is contained in:
Kristian Hartikainen
2019-03-08 10:59:31 -08:00
committed by Richard Liaw
parent d5fb7b70a9
commit 7e4b4822cf
1 changed files with 2 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files
python/ray/tune/logger.py
+2 -3
View File
@@ -226,14 +226,13 @@ class UnifiedLogger(Logger):
def close(self):
for _logger in self._loggers:
_logger.close()
self._log_syncer.sync_now(force=True)
self._log_syncer.sync_now(force=False)
self._log_syncer.close()
def flush(self):
for _logger in self._loggers:
_logger.flush()
self._log_syncer.sync_now(force=True)
self._log_syncer.wait()
self._log_syncer.sync_now(force=False)
def sync_results_to_new_location(self, worker_ip):
"""Sends the current log directory to the remote node.