[GitHub] [carbondata] marchpure opened a new pull request #3665: [HOTFIX] Fix CarbonHive CI Failures

[GitBox]

marchpure opened a new pull request #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665
 
 
    ### Why is this PR needed?
    In some cases, the carbonhive ci failes because of the permission issue.
   
    ### What changes were proposed in this PR?
   Give 777 permission to carbonhive warehouse dirs.
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No
   
       
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#discussion_r390859896
 
 

 ##########
 File path: examples/spark/src/test/scala/org/apache/carbondata/examplesCI/RunExamples.scala
 ##########
 @@ -120,11 +120,4 @@ class RunExamples extends QueryTest with BeforeAndAfterAll {
   test("DirectSQLExample") {
     DirectSQLExample.exampleBody(spark)
   }
-
-  test("HiveExample") {
 
 Review comment:
   instead of removing, please ignore this test and add a comment why you are ignoring.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-597574238
 
 
   Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/718/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-597575844
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2425/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598230988
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/735/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598233470
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2443/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

kunal642 commented on a change in pull request #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#discussion_r391773188
 
 

 ##########
 File path: integration/hive/src/test/java/org/apache/carbondata/hive/HiveCarbonTest.java
 ##########
 @@ -52,8 +58,9 @@ public static void setup() throws Exception {
     statement.execute("drop table if exists hive_carbon_table5");
     statement.execute("drop table if exists hive_table");
     statement.execute("drop table if exists hive_table_complex");
-    statement.execute("CREATE external TABLE hive_table_complex(arrayField  ARRAY<STRING>, mapField MAP<String, String>, structField STRUCT<city: String, pincode: int>) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ('field.delim'=',', 'collection.delim'='$', 'mapkey.delim'='@') location '/home/root1/projects/carbondata/integration/hive/src/main/resources/complex/' TBLPROPERTIES('external.table.purge'='false')");
-    statement.execute("CREATE external TABLE hive_table( shortField SMALLINT, intField INT, bigintField BIGINT , doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' location '/home/root1/projects/carbondata/integration/hive/src/main/resources/csv/' TBLPROPERTIES ('external.table.purge'='false')");
+    String csvFilePath = csvFile.getCanonicalPath().replace("\\", "/");
+    statement.execute(String.format("CREATE external TABLE hive_table_complex(arrayField  ARRAY<STRING>, mapField MAP<String, String>, structField STRUCT<city: String, pincode: int>) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ('field.delim'=',', 'collection.delim'='$', 'mapkey.delim'='@') location '%s' TBLPROPERTIES('external.table.purge'='false')", csvFilePath));
 
 Review comment:
   this table was using '/resources/complex' as the location..now it is pointes to /csv/ path..Please fix this

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598328637
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/739/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598329767
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2447/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on a change in pull request #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#discussion_r391841990
 
 

 ##########
 File path: integration/hive/src/test/java/org/apache/carbondata/hive/HiveCarbonTest.java
 ##########
 @@ -52,8 +58,9 @@ public static void setup() throws Exception {
     statement.execute("drop table if exists hive_carbon_table5");
     statement.execute("drop table if exists hive_table");
     statement.execute("drop table if exists hive_table_complex");
-    statement.execute("CREATE external TABLE hive_table_complex(arrayField  ARRAY<STRING>, mapField MAP<String, String>, structField STRUCT<city: String, pincode: int>) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ('field.delim'=',', 'collection.delim'='$', 'mapkey.delim'='@') location '/home/root1/projects/carbondata/integration/hive/src/main/resources/complex/' TBLPROPERTIES('external.table.purge'='false')");
-    statement.execute("CREATE external TABLE hive_table( shortField SMALLINT, intField INT, bigintField BIGINT , doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT) ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' location '/home/root1/projects/carbondata/integration/hive/src/main/resources/csv/' TBLPROPERTIES ('external.table.purge'='false')");
+    String csvFilePath = csvFile.getCanonicalPath().replace("\\", "/");
+    statement.execute(String.format("CREATE external TABLE hive_table_complex(arrayField  ARRAY<STRING>, mapField MAP<String, String>, structField STRUCT<city: String, pincode: int>) ROW FORMAT SERDE 'org.apache.hadoop.hive.contrib.serde2.MultiDelimitSerDe' WITH SERDEPROPERTIES ('field.delim'=',', 'collection.delim'='$', 'mapkey.delim'='@') location '%s' TBLPROPERTIES('external.table.purge'='false')", csvFilePath));
 
 Review comment:
   solved

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598415315
 
 
   Build Success with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/740/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598416889
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2448/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598563175
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat commented on issue #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665#issuecomment-598564510
 
 
   Merging it, Better to raise defect JIRA from next time.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3665: [HOTFIX] Fix CarbonHive CI Failures
URL: https://github.com/apache/carbondata/pull/3665
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services