[core] Option to fallback to LRU on OutOfMemory (#7410)

* Add a test for LRU fallback

* Update error message

* Upgrade arrow to master

* Integrate with arrow

* Revert "Bazel mirrors (#7385)"

This reverts commit 44aded5272.

* Don't LRU evict

* Revert "Revert "Bazel mirrors (#7385)""

This reverts commit b6359fea78d1bd3925452ca88ac71e0c9e5c7dd3.

* Add lru_evict flag

* fix internal config

* Fix

* upgrade arrow

* debug

* Set free period in config for lru_evict, override max retries to fix
test

* Fix test?

* fix test

* Revert "debug"

This reverts commit 98f01c63a267f38218f5047b1866e4c1c8280017.

* fix exception str

* Fix ref count test

* Shorten travis test?
This commit is contained in:
Stephanie Wang
2020-03-14 11:28:43 -07:00
committed by GitHub
parent 52cf77f5a9
commit 53549314c5
10 changed files with 123 additions and 36 deletions
+4
View File
@@ -1,3 +1,4 @@
import json
import logging
import time
@@ -77,6 +78,9 @@ class Cluster:
"num_gpus": 0,
"object_store_memory": 150 * 1024 * 1024, # 150 MiB
}
if "_internal_config" in node_args:
node_args["_internal_config"] = json.loads(
node_args["_internal_config"])
ray_params = ray.parameter.RayParams(**node_args)
ray_params.update_if_absent(**default_kwargs)
if self.head_node is None: