[
https://issues.apache.org/jira/browse/CARBONDATA-3365?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravindra Pesala resolved CARBONDATA-3365.
-----------------------------------------
Resolution: Fixed
Fix Version/s: 1.6.0
> Support Apache arrow vector filling from carbondata SDK
> -------------------------------------------------------
>
> Key: CARBONDATA-3365
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-3365> Project: CarbonData
> Issue Type: Improvement
> Reporter: Ajantha Bhat
> Priority: Major
> Fix For: 1.6.0
>
> Time Spent: 11h 50m
> Remaining Estimate: 0h
>
> *Background:*
> As we know Apache arrow is a cross-language development platform for
> in-memory data, It specifies a standardised language-independent columnar
> memory format for flat and hierarchical data, organised for efficient
> analytic operations on modern hardware.
> So, By integrating carbon to support filling arrow vector, contents read by
> carbondata files can be used for analytics in any programming language. say
> arrow vector filled from carbon java SDK can be read by python, c, c++ and
> many other languages supported by arrow.
> This will also increase the scope for carbondata use-cases and carbondata
> can be used for various applications as arrow is integrated already with
> many query engines.
> *Implementation:*
> *Stage1:*
> After SDK reading the carbondata file, convert carbon rows and fill the
> arrow vector.
> *Stage2:*
> Deep integration with carbon vector; for this, currently carbon SDK vector
> doesn't support filling complex columns.
> After supporting this, arrow vector can be wrapped around carbon SDK vector
> for deep integration.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
