74825db804
Fix TestGcsRedisFailureDetector ( #7710 )
...
* fix test_gcs_redis_failure_detector
* fix test_gcs_redis_failure_detector
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com >
2020-03-23 22:48:53 +08:00
ZhuSenlin and GitHub
039961b63a
rename ActorTable to LogBasedActorTable and add new ActorTable ( #7643 )
2020-03-23 15:05:43 +08:00
Edward Oakes and GitHub
8b4f5a9431
Remove non-direct-call code from core worker ( #7625 )
2020-03-22 19:20:08 -05:00
Stephanie Wang and GitHub
ba86a02b37
[core] Revert lineage pinning ( #7499 ) ( #7692 )
...
* Revert "fix (#7681 )"
This reverts commit 6a12a31b2e#7499 )"
This reverts commit 014929e658
2020-03-21 18:35:43 -07:00
Zhijun Fu and GitHub
a7a5d172b1
[core] fix bug that actor tasks from reconstructed actor is ignored by scheduling queue ( #7637 )
2020-03-21 13:05:24 +08:00
Stephanie Wang and GitHub
6a12a31b2e
fix ( #7681 )
2020-03-20 18:53:28 -07:00
Stephanie Wang and GitHub
014929e658
[core] Pin lineage of plasma objects that are still in scope ( #7499 )
...
* Add a lineage_ref_count to References
* Refactor TaskManager to store TaskEntry as a struct
* Refactor to fix deadlock between TaskManager and ReferenceCounter
Add references to task specs
* Pin TaskEntries and References in the lineage of any ObjectIDs in scope
* Fix deadlock, convert num_plasma_returns to a set of object IDs
* fix unit tests
* Feature flag
* Do not release lineage for objects that were promoted to plasma
* fix build
* fix build
* Remove num executions
* Simplify num return values
* Remove unused
* doc
* Set num returns
* Move lineage pinning flag to ReferenceCounter
* comments
* Fixes
* Remove irrelevant test (replaced by ref counting tests)
2020-03-20 10:56:43 -07:00
mehrdadn and GitHub
e69664b74b
Miscellaneous Windows compatibility bugfixes ( #7658 )
...
* Windows compatibility bug fixes
* Use WSASend/WSARecv as WSASendMsg/WSARecvMsg do not work with TCP sockets
* Clean up some TODOs
* Fix duplicate compilations
* RedisAsioClient boost::asio::error::connection_reset
Co-authored-by: Mehrdad <noreply@github.com >
2020-03-19 19:32:53 -07:00
Stephanie Wang and GitHub
c7cae036c3
[core] Only drain references for non-actor workers on shutdown ( #7668 )
...
* Only drain ref counter for non-actor tasks
* Don't force kill actors that have gone out of scope
2020-03-19 18:46:16 -07:00
fangfengbin and GitHub
0d0a41f598
[GCS]Tie lifecycle of gcs service and redis together ( #7601 )
2020-03-19 19:52:35 +08:00
Stephanie Wang and GitHub
b499100a88
Enable distributed ref counting by default ( #7628 )
...
* enable
* Turn on eager eviction
* Shorten tests and drain ReferenceCounter
* Don't force kill actor handles that have gone out of scope, lint
* Fix locks
* Cleanup Plasma Async Callback (#7452 )
* [rllib][tune] fix some nans (#7611 )
* Change /tmp to platform-specific temporary directory (#7529 )
* [Serve] UI Improvements (#7569 )
* bugfix about test_dynres.py (#7615 )
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com >
* Java call Python actor method use actor.call (#7614 )
* bug fix about useage of absl::flat_hash_map::erase and absl::flat_hash_set::erase (#7633 )
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com >
* [Java] Make both `RayActor` and `RayPyActor` inheriting from `BaseActor` (#7462 )
* [Java] Fix the issue that the cached value in `RayObject` is serialized (#7613 )
* Add failure tests to test_reference_counting (#7400 )
* Fix typo in asyncio documentation (#7602 )
* Fix segfault
* debug
* Force kill actor
* Fix test
2020-03-18 22:39:21 -07:00
Stephanie Wang and GitHub
35a4bfc885
[core] Fix leak for subscribing to object dependencies in NodeManager ( #7630 )
...
* Fix GetDependencies
* lint
2020-03-18 11:01:29 -07:00
Eric Liang and GitHub
745b9d643d
First pass at ray memory command for memory debugging ( #7589 )
2020-03-17 20:45:07 -07:00
Edward Oakes and GitHub
c1b0f9ccdf
Add failure tests to test_reference_counting ( #7400 )
2020-03-17 10:30:21 -05:00
dfa5d9b8e9
bug fix about useage of absl::flat_hash_map::erase and absl::flat_hash_set::erase ( #7633 )
...
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com >
2020-03-17 19:39:56 +08:00
ffa9df4683
bugfix about test_dynres.py ( #7615 )
...
Co-authored-by: senlin.zsl <senlin.zsl@antfin.com >
2020-03-17 13:58:44 +08:00
mehrdadn and GitHub
a0700e2f86
Change /tmp to platform-specific temporary directory ( #7529 )
2020-03-16 18:10:14 -07:00
fangfengbin and GitHub
6b37be9677
[GCS]Add job id when operating gcs table ( #7592 )
2020-03-15 12:04:04 +08:00
Kai Yang and GitHub
630e48967d
[Java] Allow passing internal config from raylet to Java worker ( #7532 )
2020-03-15 12:03:38 +08:00
a87199d240
Fix cyclic dependency between ray/util and ray/common ( #7581 )
...
* Fix cyclic dependency
Headers in ray/util should not depend on those in ray/common
* Move random generations to ray/common/test_util.h
* Add license header
Co-authored-by: Mehrdad <noreply@github.com >
Co-authored-by: Philipp Moritz <pcmoritz@gmail.com >
2020-03-14 12:44:53 -07:00
Stephanie Wang and GitHub
53549314c5
[core] Option to fallback to LRU on OutOfMemory ( #7410 )
...
* Add a test for LRU fallback
* Update error message
* Upgrade arrow to master
* Integrate with arrow
* Revert "Bazel mirrors (#7385 )"
This reverts commit 44aded5272#7385 )""
This reverts commit b6359fea78d1bd3925452ca88ac71e0c9e5c7dd3.
* Add lru_evict flag
* fix internal config
* Fix
* upgrade arrow
* debug
* Set free period in config for lru_evict, override max retries to fix
test
* Fix test?
* fix test
* Revert "debug"
This reverts commit 98f01c63a267f38218f5047b1866e4c1c8280017.
* fix exception str
* Fix ref count test
* Shorten travis test?
2020-03-14 11:28:43 -07:00
Kai Yang and GitHub
d6e8f47065
Add a flag to disable reconstruction for a killed actor ( #7346 )
2020-03-13 19:10:21 +08:00
Qing Wang and GitHub
f4656d8cc3
[Java] Enable direct call by default. ( #7408 )
...
* WIP
* Address comments.
* Linting
* Fix
* Fix
* Fix test
* Fix
* Fix single process ci
* Fix ut
* Update java/test/src/main/java/org/ray/api/test/PlasmaFreeTest.java
* Address comments
* Fix linting
* Minor update comments.
* Fix streaming CI
2020-03-13 12:25:30 +08:00
micafan and GitHub
cc91ed57dc
[core] Fix losing task state when giving up forward task. ( #7525 )
...
* fix NodeManager::Forward task bug on error
* fix lint
* revert spillback task forward
2020-03-13 11:49:44 +08:00
Edward Oakes and GitHub
768d0b3b3f
Allocate a buffer of 100 calls for each RPC handler ( #7573 )
2020-03-12 12:05:30 -07:00
ZhuSenlin and GitHub
b663bc6d67
Use gcs server to replace raylet monitor when RAY_GCS_SERVICE_ENABLED=true ( #7166 )
2020-03-12 22:13:56 +08:00
4c834b9d68
Fix the issue that gcs service client ignores error status code ( #7539 )
...
* add gcs reply status
* rebase master
* use macro to simplify
* convert status in gcs rpc client
* define a Status message in probobuf
Co-authored-by: 灵洵 <fengbin.ffb@antfin.com >
2020-03-12 15:08:29 +08:00
Stephanie Wang and GitHub
fdb528514b
[core] Ref counting for actor handles ( #7434 )
...
* tmp
* Move Exit handler into CoreWorker, exit once owner's ref count goes to 0
* fix build
* Remove __ray_terminate__ and add test case for distributed ref counting
* lint
* Remove unused
* Fixes for detached actor, duplicate actor handles
* Remove unused
* Remove creation return ID
* Remove ObjectIDs from python, set references in CoreWorker
* Fix crash
* Fix memory crash
* Fix tests
* fix
* fixes
* fix tests
* fix java build
* fix build
* fix
* check status
* check status
2020-03-10 17:45:07 -07:00
Edward Oakes and GitHub
119a303ea0
Remove static concurrency limit from gRPC server ( #7544 )
2020-03-10 16:27:02 -07:00
Edward Oakes and GitHub
dbbf0c0e70
Add Apache 2 license to C++ files ( #7520 )
2020-03-10 16:07:17 -07:00
fangfengbin and GitHub
fa785a2ad2
ServiceBasedGcsClient support detect gcs server availability and retry ( #7292 )
2020-03-10 21:01:07 +08:00
mehrdadn and GitHub
fc76586518
Redis on Windows ( #7509 )
...
* Switch hiredis on Windows to that of the Windows port of Redis
* Use boost::asio::ip::tcp::socket::native_handle_type
* Use normal hiredis instead of Windows-specific one
* Finish up using normal hiredis
Co-authored-by: Mehrdad <noreply@github.com >
2020-03-09 18:49:54 -07:00
Edward Oakes and GitHub
b4e2d5317e
Remove experimental.NoReturn ( #7475 )
2020-03-09 11:09:36 -07:00
Stephanie Wang and GitHub
95bb0c5357
Upgrade plasma to latest version, use synchronous Seal ( #7470 )
...
* Upgrade arrow to master
* fix build
* todo
* lint
* Fix hanging test
2020-03-09 10:30:44 -07:00
Edward Oakes and GitHub
0abcca258f
Add entries to in-memory store on Put() ( #7085 )
2020-03-04 10:17:27 -08:00
ijrsvt and GitHub
fb76092d75
Re-route asyncio plasma code path through raylet instead of direct plasma connection ( #7234 )
2020-03-03 15:43:46 -05:00
fangfengbin and GitHub
f5b1062ed9
Fix TwoNodeTest.TestActorTaskCrossNodes testcase when enable gcs service ( #7416 )
2020-03-03 19:37:38 +08:00
ijrsvt and GitHub
584645cc7d
Fix Experimental Async API ( #7391 )
2020-03-02 22:24:20 -06:00
2771af1036
Fix the bug of unregistered workers in worker pool ( #7343 )
...
* Fix
* Fix
* Fix complie
* Fix lint
* Fix linting
* Fix testDeleteObject
* Fix linting
* Update src/ray/raylet/worker_pool.cc
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update src/ray/raylet/worker_pool.cc
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update src/ray/raylet/worker_pool.h
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Update src/ray/raylet/worker_pool.cc
Co-Authored-By: Hao Chen <chenh1024@gmail.com >
* Address comments.
* FIx linting
Co-authored-by: Hao Chen <chenh1024@gmail.com >
2020-03-02 16:30:39 +08:00
mehrdadn and GitHub
5fb5be0ba5
Some bug fixes for Windows ( #7374 )
...
* Fix MAP_SHARED check in sys/mman.h
* Fix missing :platform_shims dependency for ray_util
* dlmalloc patch for Arrow
2020-02-28 10:22:32 -08:00
mehrdadn and GitHub
0efaa9b310
Use Redis for Windows ( #7364 )
2020-02-28 10:18:56 -08:00
micafan and GitHub
3f8b1d2756
Fix ServiceBasedGcsGcsClientTest timing bug ( #7365 )
2020-02-28 12:01:02 -06:00
Edward Oakes and GitHub
bd9411f849
Call TriggerGlobalGC when the plasma store is full ( #7337 )
2020-02-27 11:01:49 -08:00
Edward Oakes and GitHub
55ccfb6089
Fix asyncio actor race condition ( #7335 )
2020-02-27 10:16:04 -08:00
Edward Oakes and GitHub
2ad9bc5684
Move plasma retry logic into plasma store provider ( #7328 )
2020-02-26 16:57:02 -08:00
Eric Liang and GitHub
b310661338
Add internal_api.global_gc() method, which triggers gc.collect() on all workers ( #7327 )
2020-02-26 14:09:29 -08:00
fangfengbin and GitHub
ba494b5281
Fix gcs client rpc operation disorder bug ( #7283 )
2020-02-26 19:24:24 +08:00
Stephanie Wang and GitHub
9964657815
Fix plasma bug ( #7322 )
2020-02-25 18:15:28 -08:00
Edward Oakes and GitHub
44b4394afa
Remove unused AddContainedObjectIDs ( #7323 )
2020-02-25 16:42:20 -08:00
mehrdadn and GitHub
57b33f1bed
Upgrade Boost ( #6899 )
2020-02-25 14:33:12 -08:00
