[jira] [Created] (CARBONDATA-4037) Improve the table status and segment file writing

[Akash R Nilugal (Jira)]

SHREELEKHYA GAMPA created CARBONDATA-4037:
---------------------------------------------

             Summary: Improve the table status and segment file writing
                 Key: CARBONDATA-4037
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4037
             Project: CarbonData
          Issue Type: Improvement
            Reporter: SHREELEKHYA GAMPA


Currently, we update table status and segment files multiple times for a single iud/merge/compact operation and delete the index files immediately after merge. When concurrent queries are run, there may be situations like user query is trying to access the segment index files and they are not present, which is availability issue.
 * Instead of deleting carbon index files immediately after merge, delete index files only when clean files command is executed and delete only those that have existed for more than 1 hour.
 * Generate segment file after merge index and update table status at beginning and after merge index.
order:
create table status file => index files => merge index => generate segment file => update table status



--
This message was sent by Atlassian Jira
(v8.3.4#803005)