[jira] [Created] (CARBONDATA-3641) Should improve data loading performance for partition table

[Akash R Nilugal (Jira)]

David Cai created CARBONDATA-3641:
-------------------------------------

             Summary: Should improve data loading performance for partition table
                 Key: CARBONDATA-3641
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3641
             Project: CarbonData
          Issue Type: Improvement
          Components: data-load
            Reporter: David Cai


[Background]
 # only implemented commit algorithm version 1
 # generated too many segment files during loading
 # generated too many small data files and index files

[Modification]

      1.  implemented carbon commit algorithm, avoid to move data file and index files

      2.  generate the final segment file directly

     3.   optimize global_sort to avoid small files issue



--
This message was sent by Atlassian Jira
(v8.3.4#803005)