[jira] [Created] (CARBONDATA-2258) Separate visible and invisible segments info into two files to reduce the size of tablestatus file.

[Akash R Nilugal (Jira)]

Zhichao  Zhang created CARBONDATA-2258:
------------------------------------------

             Summary: Separate visible and invisible segments info into two files to reduce the size of tablestatus file.
                 Key: CARBONDATA-2258
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2258
             Project: CarbonData
          Issue Type: Improvement
          Components: core
    Affects Versions: 1.4.0, 1.3.2
            Reporter: Zhichao  Zhang
            Assignee: Zhichao  Zhang
             Fix For: 1.4.0, 1.3.2


The size of the tablestatus file is getting larger, there are many places will scan this file and it will impact the performance of reading this file.
According to the discussion on [thread|http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/The-size-of-the-tablestatus-file-is-getting-larger-does-it-impact-the-performance-of-reading-this-fi-td41941.html], it can *append* the
invisible segment list to the file called 'tablestatus.history' when execute
command 'CLEAN FILES FOR TABLE' (in method 'SegmentStatusManager.deleteLoadsAndUpdateMetadata') every time, separate visible and invisible segments into two files(tablestatus file and tablestatus.history file).
If later it needs to support listing all segments(include visible and invisible) list when execute 'SHOW SEGMENTS FOR TABLE', it just need to read from two files.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)