[DISCUSSION] Support Standard Spark's FileFormat interface in Carbondata

[ravipesala]

Hi,

Current Carbondata has deep integration with Spark to provide optimizations
in performance and also supports features like compaction, IUD, data maps
and metadata management etc. This type of integration forces user to use
CarbonSession instance to use carbon even for read and write operations.

So I am proposing standard spark's FileFormat implementation in carbon for
simple integration with Spark. Please check the jira for the design
document.
https://issues.apache.org/jira/browse/CARBONDATA-2872

--
Thanks & Regards,
Ravindra

[Liang Chen]

HI

+1, agree to support standard spark file format interface in carbondata, it
will be significantly helpful for broadening apache carbondata's ecosystem.

Regards
Liang


ravipesala wrote

> Hi,
>
> Current Carbondata has deep integration with Spark to provide
> optimizations
> in performance and also supports features like compaction, IUD, data maps
> and metadata management etc. This type of integration forces user to use
> CarbonSession instance to use carbon even for read and write operations.
>
> So I am proposing standard spark's FileFormat implementation in carbon for
> simple integration with Spark. Please check the jira for the design
> document.
> https://issues.apache.org/jira/browse/CARBONDATA-2872
>
> --
> Thanks & Regards,
> Ravindra

... [show rest of quote]

--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[kumarvishal09]

+1
-Regards
Kumar Vishal

On Thu, Aug 23, 2018 at 9:43 PM Liang Chen <[hidden email]> wrote:

> HI
>
> +1, agree to support standard spark file format interface in carbondata, it
> will be significantly helpful for broadening apache carbondata's ecosystem.
>
> Regards
> Liang
>
>
> ravipesala wrote
> > Hi,
> >
> > Current Carbondata has deep integration with Spark to provide
> > optimizations
> > in performance and also supports features like compaction, IUD, data maps
> > and metadata management etc. This type of integration forces user to use
> > CarbonSession instance to use carbon even for read and write operations.
> >
> > So I am proposing standard spark's FileFormat implementation in carbon
> for
> > simple integration with Spark. Please check the jira for the design
> > document.
> > https://issues.apache.org/jira/browse/CARBONDATA-2872
> >
> > --
> > Thanks & Regards,
> > Ravindra
>
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

... [show rest of quote]

[manishgupta88]

+1

Regards
Manish Gupta



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[xm_zzc]

+1, Good feature.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[David CaiQiang]

+1



-----
Best Regards
David Cai
--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[kanaka]

+ 1  for Carbon FileFormat.


Thanks for the design details.  


Regards,
Kanaka




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[aaron]

Does this means that we could call carbon in pyspark?




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/