[jira] [Created] (CARBONDATA-4044) Fix dirty data in indexfile while IUD with stale data in segment folder

[Akash R Nilugal (Jira)]

Xingjun Hao created CARBONDATA-4044:
---------------------------------------

             Summary: Fix dirty data in indexfile while IUD with stale data in segment folder
                 Key: CARBONDATA-4044
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4044
             Project: CarbonData
          Issue Type: Bug
            Reporter: Xingjun Hao


XX.mergecarbonindex and XX..segment records the indexfiles list of a segment. now, we generate xx.mergeindexfile and xx.segment  based on filter out all indexfiles(including carbonindex and mergecarbonindex), which will leading dirty data when there is stale data in segment folder.

For example, there are a stale index file in segment_0 folder, "0_1603763776.carbonindex".

While loading, a new carbonindex "0_16037752342.carbonindex" is wrote, when merge carbonindex files, we expect to only merge 0_16037752342.carbonindex, But If we filter out all carbonindex in segment folder, both "0_1603763776.carbonindex" and 0_16037752342.carbonindex will be merged and recorded into segment file.

 

While updating, there has same problem. 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)