wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-15 12:45:23 +08:00
Code Issues Packages Projects Releases Wiki Activity

Fix ObjectManager Crash (#2833)

Browse Source 
Fixes issue where object manager sometimes crashes within the `Wait` method: The issue stems from inconsistent behavior of the boost deadline timer's `cancel` method, which is invoked within `WaitComplete` to enforce exactly one `WaitComplete` invocation for each `Wait` request. The `cancel` method sometimes fails to actually prevent the timer's invocation of the provided handler with non-zero error code.
This commit is contained in:
Yuhong Guo
2018-09-16 02:14:13 -04:00
committed by Melih Elibol
parent 47d2f82c6c
commit a8248e8628
1 changed files with 10 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/ray/object_manager/object_manager.cc
+10 -1
View File
@@ -579,6 +579,13 @@ void ObjectManager::SubscribeRemainingWaitObjects(const UniqueID &wait_id) {
if (error_code.value() != 0) {
return;
}
if (active_wait_requests_.find(wait_id) == active_wait_requests_.end()) {
// When a subscription callback is triggered first, WaitComplete will be
// called. The timer may at the same time goes off and may be an
// interruption will post WaitComplete to main_service_ the second time.
// This check will avoid the duplicated call of this function.
return;
}
WaitComplete(wait_id);
});
}
@@ -586,7 +593,9 @@ void ObjectManager::SubscribeRemainingWaitObjects(const UniqueID &wait_id) {
}
void ObjectManager::WaitComplete(const UniqueID &wait_id) {
auto &wait_state = active_wait_requests_.find(wait_id)->second;
auto iter = active_wait_requests_.find(wait_id);
RAY_CHECK(iter != active_wait_requests_.end());
auto &wait_state = iter->second;
// If we complete with outstanding requests, then timeout_ms should be non-zero or -1
// (infinite wait time).
if (!wait_state.requested_objects.empty()) {