[jira] [Resolved] (CARBONDATA-3994) Skip Order by for map task if it is sort column and use limit pushdown for array_contains filter

Posted by Akash R Nilugal (Jira) on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Resolved-CARBONDATA-3994-Skip-Order-by-for-map-task-if-it-is-sort-column-and-use-limit-pushdownr-tp102681.html


     [ https://issues.apache.org/jira/browse/CARBONDATA-3994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor resolved CARBONDATA-3994.
--------------------------------------
    Fix Version/s: 2.1.0
       Resolution: Fixed

> Skip Order by for map task if it is sort column and use limit pushdown for array_contains filter
> ------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-3994
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3994
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Ajantha Bhat
>            Assignee: Ajantha Bhat
>            Priority: Major
>             Fix For: 2.1.0
>
>          Time Spent: 6h
>  Remaining Estimate: 0h
>
> When the order by column is in sort column, every map task output will be already sorted. No need to sort the data again.
> Hence skipping the order at map task by changing plan node from {{TakeOrderedAndProject}} --> {{CarbonTakeOrderedAndProjectExec}}
> Also in this scenario collecting the limit at map task and Array_contains() will use this limit value for row scan filtering to break scan once limit value is reached.
> Also added a carbon property to control this .
> {{carbon.mapOrderPushDown.<db_name>_<table_name>.column}}
> Note: later we can improve this for other filters also to use the limit value.

--
This message was sent by Atlassian Jira
(v8.3.4#803005)