[core] Option to fallback to LRU on OutOfMemory (#7410)

* Add a test for LRU fallback

* Update error message

* Upgrade arrow to master

* Integrate with arrow

* Revert "Bazel mirrors (#7385)"

This reverts commit 44aded5272.

* Don't LRU evict

* Revert "Revert "Bazel mirrors (#7385)""

This reverts commit b6359fea78d1bd3925452ca88ac71e0c9e5c7dd3.

* Add lru_evict flag

* fix internal config

* Fix

* upgrade arrow

* debug

* Set free period in config for lru_evict, override max retries to fix
test

* Fix test?

* fix test

* Revert "debug"

This reverts commit 98f01c63a267f38218f5047b1866e4c1c8280017.

* fix exception str

* Fix ref count test

* Shorten travis test?
This commit is contained in:
Stephanie Wang
2020-03-14 11:28:43 -07:00
committed by GitHub
parent 52cf77f5a9
commit 53549314c5
10 changed files with 123 additions and 36 deletions
+11 -1
View File
@@ -129,7 +129,17 @@ class ObjectStoreFullError(RayError):
This is raised if the attempt to store the object fails
because the object store is full even after multiple retries.
"""
pass
def __str__(self):
return super(ObjectStoreFullError, self).__str__() + (
"\n"
"The local object store is full of objects that are still in scope"
" and cannot be evicted. Try increasing the object store memory "
"available with ray.init(object_store_memory=<bytes>). "
"You can also try setting an option to fallback to LRU eviction "
"when the object store is full by calling "
"ray.init(lru_evict=True). See also: "
"https://ray.readthedocs.io/en/latest/memory-management.html.")
class UnreconstructableError(RayError):