[GitHub] [carbondata] marchpure opened a new pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…

[GitBox]

marchpure opened a new pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545
 
 
   …ct From a Parquet Table with Empty array/map
   
   Modification reason:
   (1) StringIndexOutOfBoundsException When Inserting Select From a Parquet Table with Empty array/map.
   (2) ArrayIndexOutOfBuoundsException When Inserting Select From a Parquet Table with a map with empty key and empty value
   (3) Result is incorrect when Inserting Select From a Parquet Table with a Struct with Empty String, The result will be null while the correct result is "".
   
   Modification content:
   (1) When the input value is ARRAY(), return EMPTY_DATA_RETURN in the FieldConverter.scala, ArrayParserImpl handle it.
   (2) When the input value is ARRAY(""), return EMPTY STRING ->"" in the FieldConverter.scala, ArrayParserImpl handle it.
   (3) When the input value is MAP("",""), return EMPTY STRING ->"" in the FieldConverter.scala, MapParserImpl handle it.
   (4) When the input value is MAP(), return EMPTY_DATA_RETURN ->"" in the FieldConverter.scala, MapParserImpl handle it.
   (5) When the input value is Struct(""), the StructParserImpl handle the EMPTY STRING
   
   Be sure to do all of the following checklist to help us incorporate
   your contribution quickly and easily:
   
    - [ ] Any interfaces changed?
   
    - [ ] Any backward compatibility impacted?
   
    - [ ] Document update required?
   
    - [ ] Testing done
           Please provide details on
           - Whether new unit test cases have been added or why no new tests are required?
           - How it is tested? Please attach test report.
           - Is it a performance related change? Please attach the performance test report.
           - Any additional information to help reviewers in testing this change.
         
    - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569563223
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569563265
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569565177
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1338/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569574297
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1348/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569574818
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1362/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#discussion_r361906739
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/StructParserImpl.java
 ##########
 @@ -59,6 +59,12 @@ public StructObject parse(Object data) {
           }
           return new StructObject(array);
         }
+      } else if (value.isEmpty()) {
+        Object[] array = new Object[1];
 
 Review comment:
   move to line 65

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#discussion_r361906790
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/StructParserImpl.java
 ##########
 @@ -59,6 +59,12 @@ public StructObject parse(Object data) {
           }
           return new StructObject(array);
         }
+      } else if (value.isEmpty()) {
+        Object[] array = new Object[1];
 
 Review comment:
   add comment for this else if branch

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on a change in pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#discussion_r361906854
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ##########
 @@ -300,6 +300,9 @@ private CarbonCommonConstants() {
 
   public static final String CARBON_SKIP_EMPTY_LINE_DEFAULT = "false";
 
+
+  public static final String EMPTY_DATA_RETURN = "!EMPTY_DATA_RETURN!";
 
 Review comment:
   add comment to describe this variable

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on a change in pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#discussion_r362137258
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/StructParserImpl.java
 ##########
 @@ -59,6 +59,12 @@ public StructObject parse(Object data) {
           }
           return new StructObject(array);
         }
+      } else if (value.isEmpty()) {
+        Object[] array = new Object[1];
 
 Review comment:
   Modified

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on a change in pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#discussion_r362137262
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/constants/CarbonCommonConstants.java
 ##########
 @@ -300,6 +300,9 @@ private CarbonCommonConstants() {
 
   public static final String CARBON_SKIP_EMPTY_LINE_DEFAULT = "false";
 
+
+  public static final String EMPTY_DATA_RETURN = "!EMPTY_DATA_RETURN!";
 
 Review comment:
   Modified

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

marchpure commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569850428
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569852276
 
 
   Build Success with Spark 2.1.0, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.1/1367/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569857629
 
 
   Build Failed with Spark 2.2.1, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.2/1377/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569858679
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1387/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

jackylk commented on issue #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545#issuecomment-569872561
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3545: [Carbondata-3631] StringIndexOutOfBoundsException When Inserting Sele…
URL: https://github.com/apache/carbondata/pull/3545
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services