wassname/ray
1
0
Fork 0
mirror of https://github.com/wassname/ray.git synced 2026-06-28 04:55:04 +08:00
Code Issues Packages Projects Releases Wiki Activity

[Streaming] Support streaming flow control (#7152)

Browse Source 
* streaming writer use event driven model.

* add RefreshChannelInfo

* fix name

* minor changes according reviewer comments

* Fix according to reviewer's comments

* fix bazel lint

* code polished

* Add more comments

* rename Stop & Start of EventQueue to Freeze and Unfreeze.

* add override

* fix

* fix return value

* support flow control

* add flow control ut in mock transfer

* minor changes according to comments

* add java and python worker adaption

Co-authored-by: wanxing <wanxing.wwx@alibaba-inc.com>
This commit is contained in:
Lingxuan Zuo
2020-02-24 23:48:04 +08:00
committed by GitHub
parent 7e115490d9
commit f995099e00
16 changed files with 410 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files
streaming/python/runtime/transfer.py
+8
View File
@@ -276,6 +276,14 @@ def _to_native_conf(conf):
if Config.STREAMING_EMPTY_MESSAGE_INTERVAL in conf:
config.empty_message_interval = \
conf[Config.STREAMING_EMPTY_MESSAGE_INTERVAL]
if Config.FLOW_CONTROL_TYPE in conf:
conf.flow_control_type = conf[Config.FLOW_CONTROL_TYPE]
if Config.WRITER_CONSUMED_STEP in conf:
conf.writer_consumed_step = \
conf[Config.WRITER_CONSUMED_STEP]
if Config.READER_CONSUMED_STEP in conf:
conf.reader_consumed_step = \
conf[Config.READER_CONSUMED_STEP]
logger.info("conf: %s", str(config))
return config.SerializeToString()