GitHub user QiangCai opened a pull request:
https://github.com/apache/carbondata/pull/2135 [CARBONDATA-2311][Streaming] Fix bug to avoid to append data to "streaming finish" segment
At the begin of each micro batch, check the status of current segment.
1. if the status is "streaming", continue to use this segment
2. if the status is "streaming finish", open new streaming segment to accept new streaming data
- [x] Any interfaces changed?
no
- [x] Any backward compatibility impacted?
no
- [x] Document update required?
yes, updated
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
small changes
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/QiangCai/carbondata streaming_finish
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2135.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2135
----
commit aa460655059fa31c862278ad8863f0f58f9b7edb
Author: QiangCai <qiangcai@...>
Date: 2018-04-03T06:32:59Z
avoid to append data to streaming finished segment
commit b434a8892a01e598b810d0c6b53730c524b1a5e1
Author: QiangCai <david.caiq@...>
Date: 2018-04-03T09:02:48Z
remove unused import
commit 8912ec5377249b75568d2b8fcf95a2d755f5a998
Author: QiangCai <david.caiq@...>
Date: 2018-04-03T09:48:33Z
update doc
----
---
Locked
