wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-08-08 11:25:28 +08:00
Code Issues Packages Projects Releases Wiki Activity

XRay Task Forwarding Milestone (#1785)

Browse Source 
Summary:
Able to run 1000 tasks with object dependencies on a set of distributed Raylets.

Raylet Changes:

Finalized ClientConnection class.
Task forwarding.
NM-to-NM heartbeats.
NM resource accounting for tasks.
Simple scheduling policy with task forwarding.
Creating and maintaining NM 2 NM long-lived connections and reusing them for task forwarding.
LineageCache Changes:

LineageCache without cleanup of tasks committed by remote nodes.
Lineage cache writeback and cleanup implementation.
ObjectManager Changes:

Object manager event loop/ClientConnection refactor.
Multithreaded object manager (disabled in this PR).
Testing Changes:

Integration tests for task submission on multiple Raylets.
Stress tests for object manager (with GCS and object store integration).


Co-authored-by: Stephanie Wang <swang@cs.berkeley.edu>
Co-authored-by: Alexey Tumanov <atumanov@gmail.com>
This commit is contained in:
Melih Elibol
2018-03-31 18:02:58 -07:00
committed by Philipp Moritz
co-authored by Stephanie Wang Alexey Tumanov
parent 40c9b9cd60
commit 6e06a9e338
91 changed files with 4888 additions and 1799 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/plasma/plasma_manager.cc
+1 -6
View File
@@ -487,13 +487,8 @@ PlasmaManagerState *PlasmaManagerState_init(const char *store_socket_name,
"plasma_manager", manager_addr, db_connect_args);
db_attach(state->db, state->loop, false);
ClientTableDataT client_info;
client_info.client_id = get_db_client_id(state->db).binary();
client_info.node_manager_address = std::string(manager_addr);
client_info.local_scheduler_port = 0;
client_info.object_manager_port = manager_port;
RAY_CHECK_OK(state->gcs_client.Connect(std::string(redis_primary_addr),
redis_primary_port, client_info));
redis_primary_port));
RAY_CHECK_OK(state->gcs_client.context()->AttachToEventLoop(state->loop));
} else {
state->db = NULL;