[GitHub] incubator-carbondata pull request #384: [CARBONDATA-488][SPARK2]add InsertIn...

[qiuchenjian-2]

GitHub user QiangCai opened a pull request:

    https://github.com/apache/incubator-carbondata/pull/384

    [CARBONDATA-488][SPARK2]add InsertInto feature for spark2

    1. add InsertInto feature for spark2
   
    2. optimize CarbonExample to use relation path
        And use InsertInto to load data
   
    Link:
    [CARBONDATA-488](https://issues.apache.org/jira/browse/CARBONDATA-488)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/QiangCai/incubator-carbondata insertinto_for_spark2

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-carbondata/pull/384.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #384
   
----
commit a1b3b2962a8e12f09cf5efabc15de071e105c885
Author: QiangCai <[hidden email]>
Date:   2016-12-02T17:53:32Z

    insertinto for spark2

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/384#discussion_r90694568
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonExample.scala ---
    @@ -17,79 +17,97 @@
     
     package org.apache.spark.sql.examples
     
    +import java.io.File
    +
    +import org.apache.commons.io.FileUtils
     import org.apache.spark.sql.SparkSession
    -import org.apache.spark.util.TableLoader
     
     import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.util.CarbonProperties
     
     object CarbonExample {
     
       def main(args: Array[String]): Unit = {
    -    // to run the example, plz change this path to your local machine path
    -    val rootPath = "/home/david/Documents/incubator-carbondata"
    +    val rootPath = new File(this.getClass.getResource("/").getPath
    +        + "../../../..").getCanonicalPath
    +    val storeLocation = s"$rootPath/examples/spark2/target/store"
    +    val warehouse = s"$rootPath/examples/spark2/target/warehouse"
    +    val metastoredb = s"$rootPath/examples/spark2/target/metastore_db"
    +
    +    // clean data folder
    +    if (true) {
    --- End diff --
   
    hard coded true?


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user QiangCai commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/384#discussion_r90745490
 
    --- Diff: examples/spark2/src/main/scala/org/apache/carbondata/examples/CarbonExample.scala ---
    @@ -17,79 +17,97 @@
     
     package org.apache.spark.sql.examples
     
    +import java.io.File
    +
    +import org.apache.commons.io.FileUtils
     import org.apache.spark.sql.SparkSession
    -import org.apache.spark.util.TableLoader
     
     import org.apache.carbondata.core.constants.CarbonCommonConstants
    +import org.apache.carbondata.core.util.CarbonProperties
     
     object CarbonExample {
     
       def main(args: Array[String]): Unit = {
    -    // to run the example, plz change this path to your local machine path
    -    val rootPath = "/home/david/Documents/incubator-carbondata"
    +    val rootPath = new File(this.getClass.getResource("/").getPath
    +        + "../../../..").getCanonicalPath
    +    val storeLocation = s"$rootPath/examples/spark2/target/store"
    +    val warehouse = s"$rootPath/examples/spark2/target/warehouse"
    +    val metastoredb = s"$rootPath/examples/spark2/target/metastore_db"
    +
    +    // clean data folder
    +    if (true) {
    --- End diff --
   
    this is easy to change to false, if developer want to keep data.


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/384
 
    LGTM


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/incubator-carbondata/pull/384
 
    CI
    http://136.243.101.176:8080/job/ApacheCarbonManualPRBuilder/735/


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/incubator-carbondata/pull/384


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [hidden email] or file a JIRA ticket
with INFRA.
---