[Streaming] Support streaming flow control (#7152)

* streaming writer use event driven model.

* add RefreshChannelInfo

* fix name

* minor changes according reviewer comments

* Fix according to reviewer's comments

* fix bazel lint

* code polished

* Add more comments

* rename Stop & Start of EventQueue to Freeze and Unfreeze.

* add override

* fix

* fix return value

* support flow control

* add flow control ut in mock transfer

* minor changes according to comments

* add java and python worker adaption

Co-authored-by: wanxing <wanxing.wwx@alibaba-inc.com>
This commit is contained in:
Lingxuan Zuo
2020-02-24 23:48:04 +08:00
committed by GitHub
parent 7e115490d9
commit f995099e00
16 changed files with 410 additions and 112 deletions
+5 -3
View File
@@ -98,11 +98,13 @@ class DataReader {
/// Notify input queues to clear data whose seq id is equal or less than offset.
/// It's used when checkpoint is done.
/// \param channel_info
/// \param offset
///
/// \param channel_info consumer's channel info
/// \param offset consumed channel offset
void NotifyConsumedItem(ConsumerChannelInfo &channel_info, uint64_t offset);
//// Notify message related channel to clear data.
void NotifyConsumed(std::shared_ptr<DataBundle> &message);
private:
/// Create channels and connect to all upstream.
StreamingStatus InitChannel();