[jira] [Updated] (CARBONDATA-2428) Support Flat folder structure in carbon.

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2428?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ravindra Pesala updated CARBONDATA-2428:
----------------------------------------
    Description:
1. Flat folder makes all carbondata files store flat under table path.
2. It is controlled through table property `flat_folder`. By default it is false.
3. It cannot be hybrid, so user cannot change the property once table created.
4. Segment file is created for each loading.And segment file is created under MetaData folder under table path.
5. Segment number is added as part of carbondata and index files.
6. All datamap files now create directly under table path with <tablepath>/<dmname>/<segment_number>/<task_name>/dm
 
IUD : It supports but list files during IUD may hit performance.
Compaction: Supports
Delete Segment : No impact
Clean files : No impact
Alter table : No impact
Pre Agg : Property need to inherited to child, so it also supports flat folder structure.
Partition : No Impact on this feature as it already has flat folder structure.
Streaming : Only during handoff it supports flat folder structure. Streaming segment location is no change.

 

  was:Currently carbondata writing happens in fixed path tablepath/Fact/Part0/Segment_NUM folder and it is not same as hive/parquet folder structure. This PR makes all files written will be inside tablepath, it does not maintain any segment folder structure. Only for partition it adds the folder.

> Support Flat folder structure in carbon.
> ----------------------------------------
>
>                 Key: CARBONDATA-2428
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2428
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: Ravindra Pesala
>            Priority: Major
>          Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> 1. Flat folder makes all carbondata files store flat under table path.
> 2. It is controlled through table property `flat_folder`. By default it is false.
> 3. It cannot be hybrid, so user cannot change the property once table created.
> 4. Segment file is created for each loading.And segment file is created under MetaData folder under table path.
> 5. Segment number is added as part of carbondata and index files.
> 6. All datamap files now create directly under table path with <tablepath>/<dmname>/<segment_number>/<task_name>/dm
>  
> IUD : It supports but list files during IUD may hit performance.
> Compaction: Supports
> Delete Segment : No impact
> Clean files : No impact
> Alter table : No impact
> Pre Agg : Property need to inherited to child, so it also supports flat folder structure.
> Partition : No Impact on this feature as it already has flat folder structure.
> Streaming : Only during handoff it supports flat folder structure. Streaming segment location is no change.
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)