[GitHub] [carbondata] akashrn5 opened a new pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2

[GitBox]

akashrn5 opened a new pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702
 
 
    ### Why is this PR needed?
    since we have removed old spark version, Still we are using adoop 2.7.2 as default.
   
    ### What changes were proposed in this PR?
   Upgrade to hadoop 3.1.1 and add maven profiles for old hadoop versions
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No(existing tests will take care)
   
       
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611541814
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/981/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611542037
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2694/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

QiangCai commented on a change in pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406230820
 
 

 ##########
 File path: processing/pom.xml
 ##########
 @@ -75,6 +75,11 @@
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
 
 Review comment:
   can you add dependency into profile hadoop-3.1

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406248182
 
 

 ##########
 File path: processing/pom.xml
 ##########
 @@ -75,6 +75,11 @@
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
 
 Review comment:
   currently there is profile for hadoop, do you want me to add a profile for it? As it was default i didnt add

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611562828
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/982/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611563516
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2695/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406248182
 
 

 ##########
 File path: processing/pom.xml
 ##########
 @@ -75,6 +75,11 @@
       <artifactId>junit</artifactId>
       <scope>test</scope>
     </dependency>
+    <dependency>
 
 Review comment:
   added

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611573732
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/984/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611574452
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2697/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611588422
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611598165
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/985/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611599041
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2698/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611639115
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/986/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611639804
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2699/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611663824
 
 
   Build Failed  with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/987/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#issuecomment-611664898
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2700/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on a change in pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406604832
 
 

 ##########
 File path: integration/hive/src/test/java/org/apache/carbondata/hive/HiveCarbonTest.java
 ##########
 @@ -65,7 +65,11 @@ public static void setup() throws Exception {
   public void verifyDataAfterLoad() throws Exception {
     statement.execute("drop table if exists hive_carbon_table4");
     statement.execute("CREATE TABLE hive_carbon_table4(shortField SMALLINT , intField INT, bigintField BIGINT , doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT) stored by 'org.apache.carbondata.hive.CarbonStorageHandler'");
-    statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    try {
+      statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    } catch (Exception ex) {
 
 Review comment:
   why insert will fail?

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3702: [CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406605719
 
 

 ##########
 File path: integration/hive/src/test/java/org/apache/carbondata/hive/HiveCarbonTest.java
 ##########
 @@ -65,7 +65,11 @@ public static void setup() throws Exception {
   public void verifyDataAfterLoad() throws Exception {
     statement.execute("drop table if exists hive_carbon_table4");
     statement.execute("CREATE TABLE hive_carbon_table4(shortField SMALLINT , intField INT, bigintField BIGINT , doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT) stored by 'org.apache.carbondata.hive.CarbonStorageHandler'");
-    statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    try {
+      statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    } catch (Exception ex) {
 
 Review comment:
   there are some issues, im just adding for my reference, not actual code

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

Indhumathi27 commented on a change in pull request #3702: [WIP][CARBONDATA-3769]Upgrade hadoop versionto 3.1.1 and add profile for 2.7.2
URL: https://github.com/apache/carbondata/pull/3702#discussion_r406606134
 
 

 ##########
 File path: integration/hive/src/test/java/org/apache/carbondata/hive/HiveCarbonTest.java
 ##########
 @@ -65,7 +65,11 @@ public static void setup() throws Exception {
   public void verifyDataAfterLoad() throws Exception {
     statement.execute("drop table if exists hive_carbon_table4");
     statement.execute("CREATE TABLE hive_carbon_table4(shortField SMALLINT , intField INT, bigintField BIGINT , doubleField DOUBLE, stringField STRING, timestampField TIMESTAMP, decimalField DECIMAL(18,2), dateField DATE, charField CHAR(5), floatField FLOAT) stored by 'org.apache.carbondata.hive.CarbonStorageHandler'");
-    statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    try {
+      statement.execute("insert into hive_carbon_table4 select * from hive_table");
+    } catch (Exception ex) {
 
 Review comment:
   okay

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services