[jira] [Assigned] (CARBONDATA-322) integrate spark 2.x

Posted by Akash R Nilugal (Jira) on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Assigned-CARBONDATA-322-integrate-spark-2-x-tp2022.html


     [ https://issues.apache.org/jira/browse/CARBONDATA-322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Fei Wang reassigned CARBONDATA-322:
-----------------------------------

    Assignee: Fei Wang

> integrate spark 2.x
> --------------------
>
>                 Key: CARBONDATA-322
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-322
>             Project: CarbonData
>          Issue Type: Bug
>          Components: spark-integration
>    Affects Versions: 0.2.0-incubating
>            Reporter: Fei Wang
>            Assignee: Fei Wang
>             Fix For: 0.3.0-incubating
>
>
> As spark 2.0 released. there are many nice features such as more efficient parser, vectorized execution, adaptive execution.
> It is good to integrate with spark 2.x
> Another side now in carbondata, spark integration is heavy coupling with spark code and the code need clean, we should redesign the spark integration, it should satisfy flowing requirement:
> 1. decoupled with spark, integrate according to spark datasource API(V2)
> 2. This integration should support vectorized carbon reader
> 3. Supoort write to carbondata from dadatrame
> ...

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)