[GitHub] [carbondata] ajantha-bhat opened a new pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps

[GitBox]

ajantha-bhat opened a new pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670
 
 
    ### Why is this PR needed?
   
   
    ### What changes were proposed in this PR?
   
       
    ### Does this PR introduce any user interface change?
    - No
    - Yes. (please explain the change and update document)
   
    ### Is any new testcase added?
    - No
    - Yes
   
       
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-598691992
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/749/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-598692409
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2457/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-599367099
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/766/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-599367173
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2474/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-599499358
 
 
   Build Failed  with Spark 2.4.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.4/774/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-599499804
 
 
   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/2481/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

VenuReddy2103 commented on a change in pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#discussion_r399940162
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
 ##########
 @@ -192,6 +196,16 @@ public SortStepRowHandler(SortParameters sortParameters) {
     return out;
   }
 
+  public Object[] convertToFlatRow(IntermediateSortTempRow sortTempRow) {
+    List<Object> out = new ArrayList<>();
 
 Review comment:
   Suggest to create with initialCapacity as the number of elements are at this point.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat commented on a change in pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#discussion_r399943668
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
 ##########
 @@ -192,6 +196,16 @@ public SortStepRowHandler(SortParameters sortParameters) {
     return out;
   }
 
+  public Object[] convertToFlatRow(IntermediateSortTempRow sortTempRow) {
+    List<Object> out = new ArrayList<>();
 
 Review comment:
   @VenuReddy2103 : This is WIP PR, I am working on it now. As effort is more.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat commented on a change in pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#discussion_r399943668
 
 

 ##########
 File path: processing/src/main/java/org/apache/carbondata/processing/loading/sort/SortStepRowHandler.java
 ##########
 @@ -192,6 +196,16 @@ public SortStepRowHandler(SortParameters sortParameters) {
     return out;
   }
 
+  public Object[] convertToFlatRow(IntermediateSortTempRow sortTempRow) {
+    List<Object> out = new ArrayList<>();
 
 Review comment:
   @VenuReddy2103 : This is WIP PR, I am **NOT** working on it now. As effort is more.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat commented on issue #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670#issuecomment-605799396
 
 
   This is WIP PR, **I am NOT working on it now.** as it take so much time to finish and stabilize it

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

ajantha-bhat closed pull request #3670: [WIP] Optimize write step by skipping convert to 3 steps
URL: https://github.com/apache/carbondata/pull/3670
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services