[GitHub] incubator-carbondata pull request #372: [CARBONDATA-473] integrate with spar...

[qiuchenjian-2]

GitHub user scwf opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/372

    [CARBONDATA-473] integrate with spark 2 stable datasource api

    1.  read path: based on spark2 datasource api
    2.  load path: give a tool to do data load


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/KirinKing/incubator-carbondata spark2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/372.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #372
   
----
commit a78efaea951194b1e9a3273b817d5e1641ed0f2c
Author: jackylk <[hidden email]>
Date:   2016-11-30T08:28:34Z

    add spark2 module

commit 8d536ced3c1c147bb734c769a28cebbf33f2f035
Author: jackylk <[hidden email]>
Date:   2016-11-30T08:49:03Z

    rebase

commit 308fe24500a81373dac7d87d5f2a563a35ff547d
Author: jackylk <[hidden email]>
Date:   2016-11-30T08:49:57Z

    rebase

commit d3416dc128c0d4c37d9f8191796e596d9ca7a77b
Author: jackylk <[hidden email]>
Date:   2016-11-29T12:12:22Z

    rebase

commit 120914467ee93a6613911fcf25171fa952eccd83
Author: jackylk <[hidden email]>
Date:   2016-11-30T08:50:32Z

    rebase

commit d4cee19fbd54aa008662e36479b0ee0dda0ef384
Author: QiangCai <[hidden email]>
Date:   2016-11-29T13:25:58Z

    carbon api for spark2

commit bdfa20346d2ce432f925c02c47a9d2dd7dd5d807
Author: QiangCai <[hidden email]>
Date:   2016-11-29T16:38:53Z

    fix late decoder

commit 9a96466209a5918b2fc4f55a728055ffffbc0093
Author: jackylk <[hidden email]>
Date:   2016-11-30T09:16:16Z

    fix

commit 68f61c195a90b027e9d0dbc5a87915ca68d90c00
Author: Fei Wang <[hidden email]>
Date:   2016-11-30T09:31:56Z

    Merge pull request #26 from jackylk/kingmaster
   
    Kingmaster

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    Is there an example to run carbon with spark2?
    Can these code work?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user scwf commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/372#discussion_r90207315
 
    --- Diff: integration/spark-common/pom.xml ---
    @@ -84,6 +79,11 @@
           <artifactId>junit</artifactId>
         </dependency>
         <dependency>
    +      <groupId>com.databricks</groupId>
    +      <artifactId>spark-csv_${scala.binary.version}</artifactId>
    +      <version>1.2.0</version>
    --- End diff --
   
    we can extract this as a property


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user scwf commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    the result has some issues for the carbon example:
    ```
    +----------+--------+-----------+-----------+-----------+
    |shortField|intField|bigintField|doubleField|stringField|
    +----------+--------+-----------+-----------+-----------+
    |         1|      11|       null|       44.4|      flink|
    |         3|      14|       null|       43.4|       hive|
    |         2|      10|       null|       43.4|     impala|
    |         4|      16|       null|       42.4|     impala|
    |         1|      10|       null|       48.4|      spark|
    |         5|      17|       null|       43.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    |         1|      10|       null|       47.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    |         1|      10|       null|       43.4|      spark|
    +----------+--------+-----------+-----------+-----------+
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    Yes, bigint datatype has some issue, please raise a JIRA for it.
    And can you add some basic testcase for this PR


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user scwf commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    @jacklk added test case from empty table


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    There are codestyle failing, please correct them


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    CI running
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/715/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    CI running
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/716/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    CI passed
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/717/



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/372
 
    Thanks for working for this


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/372


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---