[GitHub] [carbondata] QiangCai opened a new pull request #3897: [CARBONDATA-3958] Fix CDC merge tasks can't finish issue

[GitBox]

QiangCai opened a new pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897


    ### Why is this PR needed?
    CDC merge tasks are blocked when data loading use CarbonTableOutputFormat.getRecordWriter method.
    Because the poll method of the queue is time out in some cases.
   
    ### What changes were proposed in this PR?
    If the output is not closed, it will poll a row batch in the loops till it gets a not null batch.
    If the output is closed,  it will break the loop.
       
    ### Does this PR introduce any user interface change?
    - No
   
    ### Is any new testcase added?
    - No
   
       
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679068472


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2106/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679068984


   Build Failed  with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3847/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

QiangCai commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679096749


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

ravipesala commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679098812


   LGTM,
   Anyway we are not using it in CDC as per the PR https://github.com/apache/carbondata/pull/3856.
   This case not supposed to happen, it means source is not writing any data for more than 5 minutes. It should be some exceptional scenarios.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679123761


   @QiangCai as @ravipesala said since we are not using this, may be once after #3856 is merged, we can check and close this i think.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679159237


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2110/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679159418


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3851/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

QiangCai commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-679445110


   @ravipesala @akashrn5 we can merge it also. In the future,  other cases can use CarbonTableOutputFormat and this issue will not happen.


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-680037073


   retest this please


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-680106295


   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/3864/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

CarbonDataQA1 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-680108770


   Build Success with Spark 2.4.5, Please check CI http://121.244.95.60:12545/job/ApacheCarbon_PR_Builder_2.4.5/2123/
   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

akashrn5 commented on pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897#issuecomment-680113725


   LGTM


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]

[GitBox]

asfgit closed pull request #3897:
URL: https://github.com/apache/carbondata/pull/3897


   


----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
[hidden email]