GitHub user sounakr opened a pull request:
https://github.com/apache/carbondata/pull/1468 [WIP] Spark-2.2 Carbon Integration - Phase 1
Spark-2.2 Carbon Integration.
Phase 1 - Compilation ready for Spark-2.2.
Phase 2 - Merge the changes of Spark-2.2 and Spark-2.1 to Spark-2 folder.
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/sounakr/incubator-carbondata spark-2.2-integration
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1468.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1468
----
commit 5936e7fb25c9d5125a5863aa7648cffe5a1d8f7c
Author: QiangCai <
[hidden email]>
Date: 2017-10-27T08:06:06Z
[CARBONDATA-1628][Streaming] Re-factory LoadTableCommand to reuse code for streaming ingest in the future
Re-factory LoadTableCommand to reuse code for streaming ingest in the future
This closes #1439
commit 8e6b0a7ce9ec9b5c2e9dc8e288f148d567224e28
Author: chenliang613 <
[hidden email]>
Date: 2017-10-29T19:30:56Z
[CARBONDATA-1599] Update pull request checklist
Remove title and description items from pull request checklist, add them to How to contribute document
This closes #1451
commit 9955bed24447034f04f291dbdb2e1446e51ad8f1
Author: Zhang Zhichao <
[hidden email]>
Date: 2017-10-31T06:09:30Z
[CARBONDATA-1659] Remove spark 1.x info
This closes #1456
commit b49160935a7c3c8fe1899e3e1c49ba7022cb0938
Author: kumarvishal <
[hidden email]>
Date: 2017-10-30T15:22:19Z
[CARBONDATA-1658] Fixed Thread leak issue in no sort
Problem: In case of no sort executor service is not shutting down in writer step which is causing thread leak. In case of long run it will throwing OOM error
Solution:: Need to shutdown executor service in all the case success and failure
This closes #1454
commit 0586146a8bd953db63e1d99608ba8a77a9f5a899
Author: ravipesala <
[hidden email]>
Date: 2017-10-25T05:43:22Z
[CARBONDATA-1617] Merging carbonindex files within segment
Merge the carbonindex files after data load, so that we can reduce the IO calls to namenode and improves the read performance for first query
This closes #1436
commit e9454499dcb89ed69b1e18f79b3003ea8e5d8d25
Author: wyp <
[hidden email]>
Date: 2017-10-30T04:49:53Z
[CARBONDATA-1593] Add partition to table cause NoSuchTableException
AlterTableSplitCarbonPartition's processSchema method doesn't provide db info to sparkSession.catalog.refreshTable, this will cause NoSuchTableException when we add partitions to carbondata table.
This closes #1452
commit f209e8ee315a272f1f60a7a037d6c15fc08b6add
Author: Jacky Li <
[hidden email]>
Date: 2017-10-31T18:48:47Z
[CARBONDATA-1594] Add precision and scale to DecimalType
Refactor on DecimalType to include precision and scale parameter.
Precision and scale parameter is required for Decimal data type. In earlier code, they are stored in following classes:
ColumnSpec
ColumnPageEncoderMeta
PrimitivePageStatsCollector
ColumnSchema
Since now we have changed DataType from enum to class, precision and scale should be stored in DecimalType object only. The PR does this change.
No new test case is added in this PR since no functionality change.
This closes #1417
commit f812e41c5333ad54449d3b514becd0fddb9c5024
Author: lishuming <
[hidden email]>
Date: 2017-11-02T15:47:05Z
[Tests] Fix BTreeBlockFinderTest variable mistake
There are some obvious spelling mistake leading to some variable unused, and it may change the unit test's meaning
This closes #1462
commit 6f6897191819994da2066584721c462f03184cc6
Author: Jacky Li <
[hidden email]>
Date: 2017-11-04T10:53:46Z
[CARBONDATA-1667] Remove direct load related code
This closes #1465
commit d1cf47753a765e3f46c3a32362a844329cc1173c
Author: sounakr <
[hidden email]>
Date: 2017-11-06T07:21:17Z
Spark-2.2 Carbon Integration
----
---
Locked
