Allow remote functions to specify max executions and kill worker once limit is reached. (#660)

* implement restarting workers after certain number of task executions

* Clean up python code.

* Don't start new worker when an actor disconnects.

* Move wait_for_pid_to_exit to test_utils.py.

* Add test.

* Fix linting errors.

* Fix linting.

* Fix typo.
This commit is contained in:
Philipp Moritz
2017-06-13 00:34:58 -07:00
committed by Robert Nishihara
parent 4374ad1453
commit 54925996ca
9 changed files with 214 additions and 116 deletions
@@ -6,9 +6,9 @@ import os
import time
import ray
from ray.test.multi_node_tests import (_wait_for_nodes_to_join,
_broadcast_event,
_wait_for_event)
from ray.test.test_utils import (_wait_for_nodes_to_join,
_broadcast_event,
_wait_for_event)
# This test should be run with 5 nodes, which have 0, 0, 5, 6, and 50 GPUs for
# a total of 61 GPUs. It should be run with a large number of drivers (e.g.,