Re: Clean files enhancement

Posted by David CaiQiang on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Clean-files-enhancement-tp100088p100415.html

1. cleaning the in_progressing segment is very dangerous, please remove this
part from code.  After the user explicitly uses clean file command with an
option "clean_in_progressing"="true", we check segment lock to clean
segment.

2. if the status of a semgent is mark_for_delete/compacted, we can delete
the segment directly without backup.

3. remove code which clean stale data and partial data from
loading/compaction/update/delete feature and so on. better to use a uuid as
segment folder, Let cleaning stale data to be an optional operation. if we
don't clean stale data, table also can work fine.

5. trash folder can be under the table path.  each table has a separate
trash folder. if we clean uncertain data, we can use trash folder to store
data and use a separate folder for each transcation.



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/