Enable direct calls by default (#6367)

* wip

* add

* timeout fix

* const ref

* comments

* fix

* fix

* Move actor state into actor handle

* comments 2

* enable by default

* temp reorder

* some fixes

* add debug code

* tmp

* fix

* wip

* remove dbg

* fix compile

* fix

* fix check

* remove non direct tests

* Increment ref count before resolving value

* rename

* fix another bug

* tmp

* tmp

* Fix object pinning

* build change

* lint

* ActorManager

* tmp

* ActorManager

* fix test component failures

* Remove old code

* Remove unused

* fix

* fix

* fix resources

* fix advanced

* eric's diff

* blacklist

* blacklist

* cleanup

* annotate

* disable tests for now

* remove

* fix

* fix

* clean up verbosity

* fix test

* fix concurrency test

* Update .travis.yml

* Update .travis.yml

* Update .travis.yml

* split up analysis suite

* split up trial runner suite

* fix detached direct actors

* fix

* split up advanced tesT

* lint

* fix core worker test hang

* fix bad check fail which breaks test_cluster.py in tune

* fix some minor diffs in test_cluster

* less workers

* make less stressful

* split up test

* retry flaky tests

* remove old test flags

* fixes

* lint

* Update worker_pool.cc

* fix race

* fix

* fix bugs in node failure handling

* fix race condition

* fix bugs in node failure handling

* fix race condition

* nits

* fix test

* disable heartbeatS

* disable heartbeatS

* fix

* fix

* use worker id

* fix max fail

* debug exit

* fix merge, and apply [PATCH] fix concurrency test

* [patch] fix core worker test hang

* remove NotifyActorCreation, and return worker on completion of actor creation task

* remove actor diied callback

* Update core_worker.cc

* lint

* use task manager

* fix merge

* fix deadlock

* wip

* merge conflits

* fix

* better sysexit handling

* better sysexit handling

* better sysexit handling

* check id

* better debug

* task failed msg

* task failed msg

* retry failed tasks with delay

* retry failed tasks with delay

* clip deps

* fix

* fix core worker tests

* fix task manager test

* fix all tests

* cleanup

* set to 0 for direct tests

* dont check worker id for ownership rpc

* dont check worker id for ownership rpc

* debug messages

* add comment

* remove debug statements

* nit

* check worker id

* fix test

* owner

* fix tests
This commit is contained in:
Eric Liang
2019-12-13 13:58:04 -08:00
committed by GitHub
parent 3754effafc
commit be5dd8eb5e
16 changed files with 174 additions and 362 deletions
+11 -36
View File
@@ -151,19 +151,22 @@ def main():
timeit("1:1 actor calls async", actor_async, 1000)
a = Actor.options(is_direct_call=True).remote()
a = Actor.options(max_concurrency=16).remote()
def actor_concurrent():
ray.get([a.small_value.remote() for _ in range(1000)])
timeit("1:1 direct actor calls async", actor_concurrent, 1000)
timeit("1:1 actor calls concurrent", actor_concurrent, 1000)
a = Actor.options(is_direct_call=True, max_concurrency=16).remote()
n = 5000
n_cpu = multiprocessing.cpu_count() // 2
actors = [Actor._remote() for _ in range(n_cpu)]
client = Client.remote(actors)
def actor_concurrent():
ray.get([a.small_value.remote() for _ in range(1000)])
def actor_async_direct():
ray.get(client.small_value_batch.remote(n))
timeit("1:1 direct actor calls concurrent", actor_concurrent, 1000)
timeit("1:n actor calls async", actor_async_direct, n * len(actors))
n_cpu = multiprocessing.cpu_count() // 2
a = [Actor.remote() for _ in range(n_cpu)]
@@ -177,44 +180,16 @@ def main():
timeit("n:n actor calls async", actor_multi2, m * n)
n = 5000
n_cpu = multiprocessing.cpu_count() // 2
actors = [Actor._remote(is_direct_call=True) for _ in range(n_cpu)]
client = Client.remote(actors)
def actor_async_direct():
ray.get(client.small_value_batch.remote(n))
timeit("1:n direct actor calls async", actor_async_direct, n * len(actors))
clients = [Client.remote(a) for a in actors]
def actor_multi2_direct():
ray.get([c.small_value_batch.remote(n) for c in clients])
timeit("n:n direct actor calls async", actor_multi2_direct,
n * len(clients))
n = 1000
actors = [Actor._remote(is_direct_call=True) for _ in range(n_cpu)]
actors = [Actor._remote() for _ in range(n_cpu)]
clients = [Client.remote(a) for a in actors]
def actor_multi2_direct_arg():
ray.get([c.small_value_batch_arg.remote(n) for c in clients])
timeit("n:n direct actor calls with arg async", actor_multi2_direct_arg,
timeit("n:n actor calls with arg async", actor_multi2_direct_arg,
n * len(clients))
n = 1000
actors = [Actor._remote(is_direct_call=True) for _ in range(n_cpu)]
clients = [Client.remote(a) for a in actors]
def actor_multi2_direct_arg():
ray.get([c.small_value_batch_arg.remote(n) for c in clients])
timeit("multi client direct actor calls with arg async",
actor_multi2_direct_arg, n * len(clients))
if __name__ == "__main__":
main()