[GitHub] carbondata pull request #1990: [CARBONDATA-2195] Add new test case for parti...

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3767/



---

[qiuchenjian-2]

Github user anubhav100 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r172472894
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
           """.stripMargin)
         sql("insert into partitionTable select 1,'huawei','abc'")
         checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
    -    intercept[Exception]{
    +    val e = intercept[AnalysisException]{
    --- End diff --
   
    @xubo245
    sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
   
    closing single quote is missing after sql("alter table partitionTable PARTITION (email='def) rename to PARTITION (email='def)") it is going to pass anyhow
   



---

[qiuchenjian-2]

Github user anubhav100 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r172473473
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
           """.stripMargin)
         sql("insert into partitionTable select 1,'huawei','abc'")
         checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
    -    intercept[Exception]{
    +    val e = intercept[AnalysisException]{
           sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
         }
    +    //TODO: error message is improper
    +    assert(e.getMessage.contains("failure: identifier matching regex"))
    --- End diff --
   
    @xubo245
    sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
   
    closing single quote is missing after (email='def)") it is going to pass anyhow


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r172508765
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -256,9 +256,11 @@ test("Creation of partition table should fail if the colname in table schema and
           """.stripMargin)
         sql("insert into partitionTable select 1,'huawei','abc'")
         checkAnswer(sql("show partitions partitionTable"), Seq(Row("email=abc")))
    -    intercept[Exception]{
    +    val e = intercept[AnalysisException]{
           sql("alter table partitionTable PARTITION (email='abc') rename to PARTITION (email='def)")
         }
    +    //TODO: error message is improper
    +    assert(e.getMessage.contains("failure: identifier matching regex"))
    --- End diff --
   
    ok, fixed it. Thanks!


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4096/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2852/



---

[qiuchenjian-2]

Github user anubhav100 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r172795113
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -243,11 +242,12 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
     
       }
     
    -test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive"){
    -  intercept[Exception]{
    -    sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
    +  test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive") {
    +    //TODO: error message is improper, it is different between spark2.1 and spark2.2
    +    intercept[AnalysisException] {
    +      sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
    --- End diff --
   
    i fixed this test case with proper exception checking logic for both spark version and comments in pr #2034


---

[qiuchenjian-2]

Github user akashrn5 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r173064349
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionBadRecordLoggerTest.scala ---
    @@ -53,8 +47,7 @@ class StandardPartitionBadRecordLoggerTest extends QueryTest with BeforeAndAfter
             " ',', 'QUOTECHAR'= '\"')")
         checkAnswer(
           sql("select count(*) from sales"),
    -      Seq(Row(2)
    --- End diff --
   
    can move checkAnswer to single line in all places


---

[qiuchenjian-2]

Github user akashrn5 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r173067874
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -189,16 +189,15 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
       test("badrecords on partition column") {
         sql("create table badrecordsPartition(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
         sql(s"load data local inpath '$resourcesPath/data_partition_badrecords.csv' into table badrecordsPartition options('bad_records_action'='force')")
    -    sql("select count(*) from badrecordsPartition").show()
         checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is null"), Seq(Row(9)))
         checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is not null"), Seq(Row(2)))
       }
     
       test("badrecords fail on partition column") {
         sql("create table badrecordsPartitionfail(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
    +    //TODO: check message
    --- End diff --
   
    you can remove todo and check for the standard bad record message from exception


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r173706497
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionBadRecordLoggerTest.scala ---
    @@ -53,8 +47,7 @@ class StandardPartitionBadRecordLoggerTest extends QueryTest with BeforeAndAfter
             " ',', 'QUOTECHAR'= '\"')")
         checkAnswer(
           sql("select count(*) from sales"),
    -      Seq(Row(2)
    --- End diff --
   
    you mean in other class?


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r173708980
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -189,16 +189,15 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
       test("badrecords on partition column") {
         sql("create table badrecordsPartition(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
         sql(s"load data local inpath '$resourcesPath/data_partition_badrecords.csv' into table badrecordsPartition options('bad_records_action'='force')")
    -    sql("select count(*) from badrecordsPartition").show()
         checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is null"), Seq(Row(9)))
         checkAnswer(sql("select count(*) cnt from badrecordsPartition where intfield2 is not null"), Seq(Row(2)))
       }
     
       test("badrecords fail on partition column") {
         sql("create table badrecordsPartitionfail(intField1 int, stringField1 string) partitioned by (intField2 int) stored by 'carbondata'")
    +    //TODO: check message
    --- End diff --
   
    this issue need to be discussed, should throw BadRecordFoundException and check message. you can check the ignore test case


---

[qiuchenjian-2]

Github user xubo245 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1990#discussion_r173709334
 
    --- Diff: integration/spark-common-test/src/test/scala/org/apache/carbondata/spark/testsuite/standardpartition/StandardPartitionTableQueryTestCase.scala ---
    @@ -243,11 +242,12 @@ class StandardPartitionTableQueryTestCase extends QueryTest with BeforeAndAfterA
     
       }
     
    -test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive"){
    -  intercept[Exception]{
    -    sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
    +  test("Creation of partition table should fail if the colname in table schema and partition column is same even if both are case sensitive") {
    +    //TODO: error message is improper, it is different between spark2.1 and spark2.2
    +    intercept[AnalysisException] {
    +      sql("CREATE TABLE uniqdata_char2(name char,id int) partitioned by (NAME char)stored by 'carbondata' ")
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3844/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4192/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2945/



---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    retest this please


---

[qiuchenjian-2]

Github user xubo245 commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    retest sdv please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4202/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2957/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1990
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3853/



---