[jira] [Updated] (CARBONDATA-2853) Add min/max index for streaming segment

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

QiangCai updated CARBONDATA-2853:
---------------------------------
    Description: Streaming index file in stream segment adds min/max meta index for each streaming file during streaming ingestion. So the filter query can use the min/max index to prune the streaming files to reduce the number of the spark tasks in the driver side. Streaming file adds min/max into the blocklet header, so the filter query can skip data during scanning file.  (was: Streaming index file in stream segment adds min/max meta index for each streaming file during streaming ingestion. So the filter query can use the file-level min/max index to prune the streaming files to reduce the number of the spark tasks.)
        Summary: Add min/max index for streaming segment  (was: Add file-level min/max index for streaming segment)

> Add min/max index for streaming segment
> ---------------------------------------
>
>                 Key: CARBONDATA-2853
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2853
>             Project: CarbonData
>          Issue Type: Sub-task
>    Affects Versions: 1.5.0
>            Reporter: QiangCai
>            Assignee: QiangCai
>            Priority: Major
>             Fix For: 1.5.0
>
>         Attachments: streaming_minmax_v2.pdf
>
>          Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> Streaming index file in stream segment adds min/max meta index for each streaming file during streaming ingestion. So the filter query can use the min/max index to prune the streaming files to reduce the number of the spark tasks in the driver side. Streaming file adds min/max into the blocklet header, so the filter query can skip data during scanning file.

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)