Make Monitor remove dead Redis entries from exiting drivers. (#994)

* WIP: removing OL, OI, TT on client exit; no saving yet.

* ray_redis_module.cc: update header comment.

* Cleanup: just the removal.

* Reformat via yapf: use pep8 style instead of google.

* Checkpoint addressing comments (partially)

* Add 'b' marker before strings (py3 compat)

* Add MonitorTest.

* Use `isort` to sort imports.

* Remove some loggings

* Fix flake8 noqa marker runtest.py

* Try to separate tests out to monitor_test.py

* Rework cleanup algorithm: correct logic

* Extend tests to cover multi-shard cases

* Add some small comments and formatting changes.
This commit is contained in:
Zongheng Yang
2017-09-26 00:11:38 -07:00
committed by Robert Nishihara
parent 6e9657e696
commit 5a50e80b63
7 changed files with 698 additions and 174 deletions
+19 -16
View File
@@ -8,22 +8,25 @@
#include "common_protocol.h"
/**
* Various tables are maintained in redis:
*
* == OBJECT TABLE ==
*
* This consists of two parts:
* - The object location table, indexed by OL:object_id, which is the set of
* plasma manager indices that have access to the object.
* - The object info table, indexed by OI:object_id, which is a hashmap with key
* "hash" for the hash of the object and key "data_size" for the size of the
* object in bytes.
*
* == TASK TABLE ==
*
* TODO(pcm): Fill this out.
*/
// Various tables are maintained in redis:
//
// == OBJECT TABLE ==
//
// This consists of two parts:
// - The object location table, indexed by OL:object_id, which is the set of
// plasma manager indices that have access to the object.
// (In redis this is represented by a zset (sorted set).)
//
// - The object info table, indexed by OI:object_id, which is a hashmap of:
// "hash" -> the hash of the object,
// "data_size" -> the size of the object in bytes,
// "task" -> the task ID that generated this object.
// "is_put" -> 0 or 1.
//
// == TASK TABLE ==
//
// TODO(pcm): Fill this out.
//
#define OBJECT_INFO_PREFIX "OI:"
#define OBJECT_LOCATION_PREFIX "OL:"