[GitHub] carbondata pull request #1833: [CARBONDATA-2036] Fix the insert static parti...

[qiuchenjian-2]

GitHub user ravipesala opened a pull request:

    https://github.com/apache/carbondata/pull/1833

    [CARBONDATA-2036] Fix the insert static partition with integer values prefix with 0 not working

    When trying to insert overwrite on the static partition with 0 at first on int column has an issue.
    Example :
    create table test(d1 string) partition by (c1 int, c2 int, c3 int)
    And use insert overwrite table partition(01, 02, 03) select "s1"
     
    The above case has a problem as 01 is not converting to an actual integer to partition map file.
    Solution :
    Convert the partition values to corresponding datatype value before adding to partition file.
   
    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [X] Any interfaces changed?
     
     - [X] Any backward compatibility impacted?
     
     - [X] Document update required?
   
     - [X] Testing done
           Tests added
           
     - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ravipesala/incubator-carbondata static-partition-int-issue

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1833.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1833
   
----
commit e9a925e36f8e44f6066853542cef3487dffec63c
Author: ravipesala <ravi.pesala@...>
Date:   2018-01-18T13:21:50Z

    Fix the insert static partition with integer values prefix with 0 not working

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2960/



---

[qiuchenjian-2]

Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1833#discussion_r162419365
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala ---
    @@ -243,7 +243,7 @@ case class CarbonLoadDataCommand(
           } catch {
             case CausedBy(ex: NoRetryException) =>
               // update the load entry in table status file for changing the status to marked for delete
    -          if (isUpdateTableStatusRequired) {
    +          if (isUpdateTableStatusRequired && !table.isHivePartitionTable) {
    --- End diff --
   
    This check is not required because isUpdateTableStatusRequired  flag will be set to true only if the table is not a partition table


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1732/



---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1833#discussion_r162545921
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/management/CarbonLoadDataCommand.scala ---
    @@ -243,7 +243,7 @@ case class CarbonLoadDataCommand(
           } catch {
             case CausedBy(ex: NoRetryException) =>
               // update the load entry in table status file for changing the status to marked for delete
    -          if (isUpdateTableStatusRequired) {
    +          if (isUpdateTableStatusRequired && !table.isHivePartitionTable) {
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2989/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2975/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1745/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1754/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2984/



---

[qiuchenjian-2]

Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1833#discussion_r162600231
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableOverwriteTestCase.scala ---
    @@ -155,9 +155,28 @@ class StandardPartitionTableOverwriteTestCase extends QueryTest with BeforeAndAf
         checkAnswer(sql("select count(*) from weather6"), Seq(Row(2)))
       }
     
    +  test("Test overwrite static partition with wrong int value") {
    +    sql(
    +      """
    +        | CREATE TABLE weather7 (type String)
    +        | PARTITIONED BY (year int, month int, day int)
    +        | STORED BY 'org.apache.carbondata.format'
    +      """.stripMargin)
    +
    +    sql("insert into weather7 partition(year=2014, month=05, day=25) select 'rainy'")
    +    sql("insert into weather7 partition(year=2014, month=04, day=23) select 'cloudy'")
    +    sql("insert overwrite table weather7 partition(year=2014, month=05, day=25) select 'sunny'")
    +    checkExistence(sql("select * from weather7"), true, "sunny")
    +    checkAnswer(sql("select count(*) from weather7"), Seq(Row(2)))
    +    sql("insert into weather7 partition(year=2014, month, day) select 'rainy1',06,25")
    +    sql("insert into weather7 partition(year=2014, month=01, day) select 'rainy2',27")
    +    sql("insert into weather7 partition(year=2014, month=01, day=02) select 'rainy3'")
    +    checkAnswer(sql("select count(*) from weather7 where month=1"), Seq(Row(2)))
    +  }
    +
     
       override def afterAll = {
    -    dropTable
    +//    dropTable
    --- End diff --
   
    Please uncomment this line


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2997/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1757/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2987/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2989/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1759/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3001/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3004/



---

[qiuchenjian-2]

Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    retest sdv please


---

[qiuchenjian-2]

Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/1833
 
    retest this please


---