GitHub user xubo245 opened a pull request:
https://github.com/apache/carbondata/pull/3051 [CARBONDATA-3221] Fix the error of SDK don't support read multiple file from S3 SDK reader is ok with filter, but when we read data without filter, the ((CarbonInputSplit) inputSplit).getDetailInfo().getBlockFooterOffset() will be 0 and FileReader reader don't closed after readByteBuffer in org.apache.carbondata.hadoop.util.CarbonVectorizedRecordReader#initialize, so we should invoke finish after readByteBuffer Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed? No - [ ] Any backward compatibility impacted? No - [ ] Document update required? No - [ ] Testing done Please provide details on - Whether new unit test cases have been added or why no new tests are required? - How it is tested? Please attach test report. - Is it a performance related change? Please attach the performance test report. - Any additional information to help reviewers in testing this change. change test case in example - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. no You can merge this pull request into a Git repository by running: $ git pull https://github.com/xubo245/carbondata CARBONDATA-3221_SDKSupportMultipleFiles Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/3051.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3051 ---- commit fd2858083f8a9041c67f4f8192096b52ed0cf0e1 Author: xubo245 <xubo29@...> Date: 2019-01-04T08:53:48Z [CARBONDATA-3221] Fix the error of SDK don't support read multiple file from S3 SDK reader is ok with filter, but when we read data without filter, the ((CarbonInputSplit) inputSplit).getDetailInfo().getBlockFooterOffset() will be 0 and FileReader reader don't closed after readByteBuffer in org.apache.carbondata.hadoop.util.CarbonVectorizedRecordReader#initialize, so we should invoke finish after readByteBuffer commit f47fc32de498b99342593518be5ffe0f08d41ead Author: xubo245 <xubo29@...> Date: 2019-01-04T09:06:06Z add config ---- --- |
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/3051 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2160/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/3051 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2162/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/3051 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2163/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/3051 Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10417/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/3051 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2376/ --- |
In reply to this post by qiuchenjian-2
Github user xubo245 commented on the issue:
https://github.com/apache/carbondata/pull/3051 @KanakaKumar @kunal642 @jackylk @QiangCai @zzcclp @ravikiran23 Please review it. --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/3051 LGTM --- |
In reply to this post by qiuchenjian-2
|
Free forum by Nabble | Edit this page |