Jacky Li created CARBONDATA-307:
-----------------------------------
Summary: Support full functionality in CarbonInputFormat
Key: CARBONDATA-307
URL:
https://issues.apache.org/jira/browse/CARBONDATA-307 Project: CarbonData
Issue Type: Improvement
Components: spark-integration
Affects Versions: 0.1.0-incubating
Reporter: Jacky Li
Fix For: 0.2.0-incubating
Currently, there are two read path in carbon-spark module:
1. CarbonContext => CarbonDatasourceRelation => CarbonScanRDD => QueryExecutor
In this case, CarbonScanRDD uses CarbonInputFormat to get the split, and use QueryExecutor for scan.
2. SqlContext => CarbonDatasourceHadoopRelation => CarbonHadoopFSRDD => CarbonRecordReader
In this case, CarbonHadoopFSRDD uses CarbonInputFormat to do both get split and scan
It create unnecessary duplicate code, they need to be unified.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)