[GitHub] carbondata pull request #2969: [CARBONDATA-3127]Fix the TestCarbonSerde exce...

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/9932/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/1884/



---

[qiuchenjian-2]

Github user SteNicholas commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    @xubo245 Please review this request again include improvement of the comments referred by @xuchuanyin .


---

[qiuchenjian-2]

Github user xuchuanyin commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    LGTM


---

[qiuchenjian-2]

Github user zzcclp commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    LGTM


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1804/



---

[qiuchenjian-2]

Github user SteNicholas commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    @xubo245 @zzcclp @xuchuanyin I have already fixed hive integration bug based on HiveExample.Please review these updates.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1805/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1806/



---

[qiuchenjian-2]

Github user SteNicholas commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    @xubo245 @zzcclp @xuchuanyin I have already fixed hive integration bug based on HiveExample.Please review these updates.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1808/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/1809/



---

[qiuchenjian-2]

Github user SteNicholas commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    @xubo245 @zzcclp @xuchuanyin I have already fixed hive integration bug based on HiveExample.Please review these updates.


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2019/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10068/



---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2969#discussion_r242389254
 
    --- Diff: integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala ---
    @@ -43,28 +43,25 @@ object HiveExample {
     
         import org.apache.spark.sql.CarbonSession._
     
    -    val carbonSession = SparkSession
    -      .builder()
    -      .master("local")
    -      .appName("HiveExample")
    -      .config("carbonSession.sql.warehouse.dir", warehouse).enableHiveSupport()
    -      .getOrCreateCarbonSession(
    -        store, metaStore_Db)
    +    val carbonSession = SparkSession.builder()
    +      .master("local").appName("HiveExample")
    --- End diff --
   
    Please keep it，no need to change


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2969#discussion_r242390063
 
    --- Diff: integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala ---
    @@ -43,28 +43,25 @@ object HiveExample {
     
         import org.apache.spark.sql.CarbonSession._
     
    -    val carbonSession = SparkSession
    -      .builder()
    -      .master("local")
    -      .appName("HiveExample")
    -      .config("carbonSession.sql.warehouse.dir", warehouse).enableHiveSupport()
    -      .getOrCreateCarbonSession(
    -        store, metaStore_Db)
    +    val carbonSession = SparkSession.builder()
    +      .master("local").appName("HiveExample")
    +      .enableHiveSupport()
    +      .config("spark.sql.warehouse.dir", warehouse)
    +      .getOrCreateCarbonSession(store, metaStore_Db)
     
         carbonSession.sql("""DROP TABLE IF EXISTS HIVE_CARBON_EXAMPLE""".stripMargin)
     
         carbonSession
           .sql(
    -        """CREATE TABLE HIVE_CARBON_EXAMPLE (ID int,NAME string,SALARY double) STORED BY
    -          |'CARBONDATA' """
    -          .stripMargin)
    +        "CREATE TABLE HIVE_CARBON_EXAMPLE (ID int,NAME string,SALARY double) " +
    +          "STORED BY 'CARBONDATA' ")
     
         carbonSession.sql(
           s"""
    -           LOAD DATA LOCAL INPATH '$rootPath/integration/hive/src/main/resources/data.csv' INTO
    -           TABLE
    -         HIVE_CARBON_EXAMPLE
    -           """)
    +               LOAD DATA LOCAL INPATH '$rootPath/integration/hive/src/main/resources/data.csv' INTO
    +               TABLE
    +             HIVE_CARBON_EXAMPLE
    --- End diff --
   
    please optimize it or revert it.


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2969#discussion_r242391201
 
    --- Diff: integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala ---
    @@ -85,19 +82,25 @@ object HiveExample {
     
         logger.info(s"============HIVE CLI IS STARTED ON PORT $port ==============")
     
    -    statement.execute("CREATE TABLE IF NOT EXISTS " + "HIVE_CARBON_EXAMPLE " +
    -                      " (ID int, NAME string,SALARY double)")
    +    statement.execute("DROP TABLE IF EXISTS HIVE_CARBON_EXAMPLE")
    +
    +    statement.execute("CREATE TABLE HIVE_CARBON_EXAMPLE " +
    +      " (ID int, NAME string,SALARY double) " +
    +      "ROW FORMAT SERDE 'org.apache.carbondata.hive.CarbonHiveSerDe' " +
    +      "WITH SERDEPROPERTIES ('mapreduce.input.carboninputformat.databaseName'='default', " +
    +      "'mapreduce.input.carboninputformat.tableName'='HIVE_CARBON_EXAMPLE')")
    +
         statement
           .execute(
             "ALTER TABLE HIVE_CARBON_EXAMPLE SET FILEFORMAT INPUTFORMAT \"org.apache.carbondata." +
    -        "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." +
    -        "MapredCarbonOutputFormat\"SERDE \"org.apache.carbondata.hive." +
    -        "CarbonHiveSerDe\" ")
    +          "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." +
    --- End diff --
   
    Can you add space before OUTPUTFORMAT?


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2969#discussion_r242391224
 
    --- Diff: integration/hive/src/main/scala/org/apache/carbondata/hiveexample/HiveExample.scala ---
    @@ -85,19 +82,25 @@ object HiveExample {
     
         logger.info(s"============HIVE CLI IS STARTED ON PORT $port ==============")
     
    -    statement.execute("CREATE TABLE IF NOT EXISTS " + "HIVE_CARBON_EXAMPLE " +
    -                      " (ID int, NAME string,SALARY double)")
    +    statement.execute("DROP TABLE IF EXISTS HIVE_CARBON_EXAMPLE")
    +
    +    statement.execute("CREATE TABLE HIVE_CARBON_EXAMPLE " +
    +      " (ID int, NAME string,SALARY double) " +
    +      "ROW FORMAT SERDE 'org.apache.carbondata.hive.CarbonHiveSerDe' " +
    +      "WITH SERDEPROPERTIES ('mapreduce.input.carboninputformat.databaseName'='default', " +
    +      "'mapreduce.input.carboninputformat.tableName'='HIVE_CARBON_EXAMPLE')")
    +
         statement
           .execute(
             "ALTER TABLE HIVE_CARBON_EXAMPLE SET FILEFORMAT INPUTFORMAT \"org.apache.carbondata." +
    -        "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." +
    -        "MapredCarbonOutputFormat\"SERDE \"org.apache.carbondata.hive." +
    -        "CarbonHiveSerDe\" ")
    +          "hive.MapredCarbonInputFormat\"OUTPUTFORMAT \"org.apache.carbondata.hive." +
    +          "MapredCarbonOutputFormat\"SERDE \"org.apache.carbondata.hive." +
    --- End diff --
   
    Can you add space before SERDE?


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/2969
 
    @SteNicholas Please optimize the title


---