[DISCUSSION] Implement file-level Min/Max index for streaming segment

[David CaiQiang]

Hi All,
    Currently, the filter queries on the streaming table always scan all
streaming files, even though there are no data in streaming files that meet
the filter conditions.
    So I try to support file-level min/max index on streaming segment. It
helps to reduce the task number and improve the performance of filter scan
in some cases.
    Please check the document in JIRA:  
https://issues.apache.org/jira/browse/CARBONDATA-2853
<https://issues.apache.org/jira/browse/CARBONDATA-2853>  
    Any question, suggestion?



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Liang Chen]

Hi

+1 for this proposal.

Regards
Liang


David CaiQiang wrote

> Hi All,
>     Currently, the filter queries on the streaming table always scan all
> streaming files, even though there are no data in streaming files that
> meet
> the filter conditions.
>     So I try to support file-level min/max index on streaming segment. It
> helps to reduce the task number and improve the performance of filter scan
> in some cases.
>     Please check the document in JIRA:  
> https://issues.apache.org/jira/browse/CARBONDATA-2853
> &lt;https://issues.apache.org/jira/browse/CARBONDATA-2853&gt; 
>     Any question, suggestion?
>
>
>
> -----
> Best Regards
> David Cai
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[sraghunandan]

+1. this would improve the query performance on streaming tables

On Sun, Aug 26, 2018 at 4:12 PM Liang Chen <[hidden email]> wrote:

> Hi
>
> +1 for this proposal.
>
> Regards
> Liang
>
>
> David CaiQiang wrote
> > Hi All,
> >     Currently, the filter queries on the streaming table always scan all
> > streaming files, even though there are no data in streaming files that
> > meet
> > the filter conditions.
> >     So I try to support file-level min/max index on streaming segment. It
> > helps to reduce the task number and improve the performance of filter
> scan
> > in some cases.
> >     Please check the document in JIRA:
> > https://issues.apache.org/jira/browse/CARBONDATA-2853
> > &lt;https://issues.apache.org/jira/browse/CARBONDATA-2853&gt;
> >     Any question, suggestion?
> >
> >
> >
> > -----
> > Best Regards
> > David Cai
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[xm_zzc]

+1.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[kumarvishal09]

+1
Regards
Kumar Vishal

On Mon, 27 Aug 2018 at 07:15, xm_zzc <[hidden email]> wrote:

> +1.
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[manishgupta88]

+1

Regards
Manish Gupta

On Mon, 27 Aug 2018 at 9:47 AM, Kumar Vishal <[hidden email]>
wrote:

> +1
> Regards
> Kumar Vishal
>
> On Mon, 27 Aug 2018 at 07:15, xm_zzc <[hidden email]> wrote:
>
> > +1.
> >
> >
> >
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> >
>

[Venkata Gollamudi]

+1

Regards,
Venkata Ramana

On Mon, Aug 27, 2018 at 2:10 PM manish gupta <[hidden email]>
wrote:

> +1
>
> Regards
> Manish Gupta
>
> On Mon, 27 Aug 2018 at 9:47 AM, Kumar Vishal <[hidden email]>
> wrote:
>
> > +1
> > Regards
> > Kumar Vishal
> >
> > On Mon, 27 Aug 2018 at 07:15, xm_zzc <[hidden email]> wrote:
> >
> > > +1.
> > >
> > >
> > >
> > > --
> > > Sent from:
> > >
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> > >
> >
>