Re: [carbondata-presto enhancements] support reading carbon SDK writer output in presto

Posted by ravipesala on Dec 11, 2018; 2:08am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/carbondata-presto-enhancements-support-reading-carbon-SDK-writer-output-in-presto-tp69978p70131.html

Hi Jacky,

In spark integration we have two approaches one with very deep integration
and one with shallow integration using the sparks fileformat. One with deep
integration we use the datasource name as carbondata, this name also
registered to java services so anything which comes with this datasource
name uses the deep integration path.
Another with shallow integration we use datasource name as carbon and
extracted this to spark-datasource module. So any table with this carbon
datasource name comes to the fileformat flow.

This datasource names are nothing do with transactional and non
transactional. It is about the spark datasource implementations. Basically I
am trying to tell is with carbondata datasource can read both transactional
and non transactional data. We introduced carbon datasource name only for
the sake of spark to identify the type of implementation flow it should
choose.


Regards,
Ravindra.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/