[GitHub] carbondata pull request #1000: [CARBONDATA-1018] Add unsafe ColumnPage imple...

[qiuchenjian-2]

GitHub user jackylk opened a pull request:

    https://github.com/apache/carbondata/pull/1000

    [CARBONDATA-1018] Add unsafe ColumnPage implementation

    This PR is based on #987
   
    In this PR, an `UnsafeColumnPage` is added, it can reduce memory requirement for data loading.
    User can add following property to enable this feature
    ```
        CarbonProperties.getInstance()
          .addProperty(CarbonCommonConstants.ENABLE_LOADING_UNSAFE_COLUMN_PAGE, "true")
    ```

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/jackylk/incubator-carbondata unsafepage

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1000.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1000
   
----
commit f3fc51e8ea81d928b2580533b968cf1553424c00
Author: jackylk <[hidden email]>
Date:   2017-06-06T07:39:05Z

    extract interface

commit aac93612dd417c2bbc5e03cd49826348aa6bde7a
Author: jackylk <[hidden email]>
Date:   2017-06-06T07:43:47Z

    add EncodingStrategy

commit be954551a4f560f3f1ecd696ac032793eac33bae
Author: jackylk <[hidden email]>
Date:   2017-06-06T11:03:44Z

    add unsafe column page

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/105/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2232/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2237/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/110/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/177/<h2>Failed Tests: <span class='status-failure'>11</span></h2><h3><a name='carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark' /><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark/testReport'>carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark</a>: <span class='status-failure'>2</span></h3><ul><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark/testReport/org.apache.carbondata.spark.util/AutoHighCardinalityIdentifyTestCase/auto_identify_high_cardinality_column_in_first_load__396/'><strong>org.apache.carbondata.spark.util.AutoHighCardinalityIdentifyTestCase.auto identify high cardinality column in first load #396</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark/testReport/org.apache.carbondata.spark.util/AutoHighCa
 rdinalityIdentifyTestCase/skip_auto_identify_high_cardinality_column_for_column_group/'><strong>org.apache.carbondata.spark.util.AutoHighCardinalityIdentifyTestCase.skip auto identify high cardinality column for column group</strong></a></li></ul><h3><a name='carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test' /><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport'>carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test</a>: <span class='status-failure'>9</span></h3><ul><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/Test_____special_character_data_loading_for_complex_types/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.Test ^ * special character data load
 ing for complex types</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/complex_filter_set1/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.complex filter set1</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/select_count_mobile__channelsId_from_complexcarbontable_group_by_mobile_channelsId/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.select count(mobile),channelsId from complexcarbontable group by mobile,channelsId</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbond
 ata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/select_count_mobile__channelsId_from_complexcarbontable_group_by_mobile_channelsId_order_by_channelsId/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.select count(mobile),channelsId from complexcarbontable group by mobile,channelsId order by channelsId</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/select_count_mobile__channelsId_from_complexcarbontable_group_by_mobile_channelsId_order_by_channelsId_limit_10/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.select count(mobile),channelsId from complexcarbontable group by mobile,channelsId order by channelsId limit 10</strong></a></li><li><a href='ht
 tps://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.integration.spark.testsuite.complexType/TestComplexTypeQuery/select_count_mobile__channelsId_from_complexcarbontable_where_MAC_0_____MAC1___group_by_mobile_channelsId_order_by_channelsId_limit_10/'><strong>org.apache.carbondata.integration.spark.testsuite.complexType.TestComplexTypeQuery.select count(mobile),channelsId from complexcarbontable where MAC[0] = 'MAC1'  group by mobile,channelsId order by channelsId limit 10</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.spark.testsuite.blockprune/CarbonCustomBlockDistributionTest/test_block_prune_query/'><strong>org.apache.carbondata.spark.testsuite.blockprune.CarbonCustomBlockDistributionTest.test block prune query</strong></a></li><li><a href='https://builds.apache.org/job/c
 arbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.spark.testsuite.detailquery/ValueCompressionDataTypeTestCase/ActualDataType_double_ChangedDatatype_Short_CompressionType_NonDecimalMaxMin/'><strong>org.apache.carbondata.spark.testsuite.detailquery.ValueCompressionDataTypeTestCase.ActualDataType:double,ChangedDatatype:Short,CompressionType:NonDecimalMaxMin</strong></a></li><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/177/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.spark.testsuite.detailquery/ValueCompressionDataTypeTestCase/ActualDataType_double_ChangedDatatype_byte_CompressionType_NonDecimalMaxMin/'><strong>org.apache.carbondata.spark.testsuite.detailquery.ValueCompressionDataTypeTestCase.ActualDataType:double,ChangedDatatype:byte,CompressionType:NonDecimalMaxMin</strong></a></li></ul>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2304/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2307/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/180/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2308/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/181/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/199/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2322/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    apache:retest this please


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2324/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/201/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/211/<h2>Failed Tests: <span class='status-failure'>1</span></h2><h3><a name='carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test' /><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/211/org.apache.carbondata$carbondata-spark-common-test/testReport'>carbondata-pr-spark-1.6/org.apache.carbondata:carbondata-spark-common-test</a>: <span class='status-failure'>1</span></h3><ul><li><a href='https://builds.apache.org/job/carbondata-pr-spark-1.6/211/org.apache.carbondata$carbondata-spark-common-test/testReport/org.apache.carbondata.spark.testsuite.allqueries/InsertIntoCarbonTableTestCase/insert_into_carbon_table_from_carbon_table_union_query/'><strong>org.apache.carbondata.spark.testsuite.allqueries.InsertIntoCarbonTableTestCase.insert into carbon table from carbon table union query</strong></a></li></ul>



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2334/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/2452/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit commented on the issue:

    https://github.com/apache/carbondata/pull/1000
 
   
    Refer to this link for build results (access rights to CI server needed):
    https://builds.apache.org/job/carbondata-pr-spark-1.6/333/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---