From fb4d9917320fe4992dc84d6fce88f10d790acecf Mon Sep 17 00:00:00 2001 From: Stephanie Wang Date: Tue, 23 Jan 2018 13:40:50 -0800 Subject: [PATCH] Add actor reconstruction limitations to documentation (#1452) * Add limitations to actor fault tolerance * rename heading --- doc/source/actors.rst | 7 ++++++- doc/source/fault-tolerance.rst | 12 +++++++++--- 2 files changed, 15 insertions(+), 4 deletions(-) diff --git a/doc/source/actors.rst b/doc/source/actors.rst index 99bbee788..c7594592f 100644 --- a/doc/source/actors.rst +++ b/doc/source/actors.rst @@ -314,10 +314,15 @@ We are working to address the following issues. 2. **Returning actor handles:** Actor handles currently cannot be returned from a remote function or actor method. Similarly, ``ray.put`` cannot be called on an actor handle. -3. **Reconstruction evicted actor objects:** If ``ray.get`` is called on an +3. **Reconstruction of evicted actor objects:** If ``ray.get`` is called on an evicted object that was created by an actor method, Ray currently will not reconstruct the object. For more information, see the documentation on `fault tolerance`_. +4. **Deterministic reconstruction of lost actors:** If an actor is lost due to + node failure, the actor is reconstructed on a new node, following the order + of initial execution. However, new tasks that are scheduled onto the actor + in the meantime may execute in between re-executed tasks. This could be an + issue if your application has strict requirements for state consistency. .. _`asynchronous parameter server example`: http://ray.readthedocs.io/en/latest/example-parameter-server.html .. _`fault tolerance`: http://ray.readthedocs.io/en/latest/fault-tolerance.html diff --git a/doc/source/fault-tolerance.rst b/doc/source/fault-tolerance.rst index 131c5dfb7..a4692f904 100644 --- a/doc/source/fault-tolerance.rst +++ b/doc/source/fault-tolerance.rst @@ -37,11 +37,11 @@ occurred after the checkpoint will be re-executed. Note that actor checkpointing is currently an experimental feature. -Unhandled Failures ------------------- +Current Limitations +------------------- At the moment, Ray does not handle all failure scenarios. We are working on -addressing these problems. +addressing these known problems. Process Failures ~~~~~~~~~~~~~~~~ @@ -58,3 +58,9 @@ Lost Objects evicted, and is later needed, Ray will not reconstruct this object. 2. If an object is constructed by an actor method, is then evicted, and is later needed, Ray will not reconstruct this object. + +Actor Reconstruction +~~~~~~~~~~~~~~~~~~~~ + +1. Actor reconstruction follows the order of initial execution, but new tasks + may get interleaved with the re-executed tasks.