[jira] [Created] (CARBONDATA-307) Support full functionality in CarbonInputFormat

Posted by Akash R Nilugal (Jira) on Oct 13, 2016; 2:55am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/jira-Created-CARBONDATA-307-Support-full-functionality-in-CarbonInputFormat-tp1831.html

Jacky Li created CARBONDATA-307:
-----------------------------------

             Summary: Support full functionality in CarbonInputFormat
                 Key: CARBONDATA-307
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-307
             Project: CarbonData
          Issue Type: Improvement
          Components: spark-integration
    Affects Versions: 0.1.0-incubating
            Reporter: Jacky Li
             Fix For: 0.2.0-incubating


Currently, there are two read path in carbon-spark module:
1. CarbonContext => CarbonDatasourceRelation => CarbonScanRDD => QueryExecutor
In this case, CarbonScanRDD uses CarbonInputFormat to get the split, and use QueryExecutor for scan.

2. SqlContext => CarbonDatasourceHadoopRelation => CarbonHadoopFSRDD => CarbonRecordReader
In this case, CarbonHadoopFSRDD uses CarbonInputFormat to do both get split and scan

It create unnecessary duplicate code, they need to be unified.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)