朱峰 created CARBONDATA-3621:
------------------------------
Summary: when set carbon.fs.custom.file.provider, only can use FileReaderImpl to read custom file.
Key: CARBONDATA-3621
URL:
https://issues.apache.org/jira/browse/CARBONDATA-3621 Project: CarbonData
Issue Type: Improvement
Components: core
Affects Versions: 1.6.1
Reporter: 朱峰
I want carbondata to support my filesystem like myfs://xxx. set carbon.fs.custom.file.provider=org.apache.carbondata.core.datastore.impl.myFSProvider and my java code like this:
public class MyFSProvider implements FileTypeInterface{
@Override public CarbonFile getCarbonFile(String path, Configuration configuration) {
if(this.isPathSupported(path)){
return new MyFSCarbonFile(path,configuration);
}
return null;
}
@Override public boolean isPathSupported(String path) {
return path.startsWith("myfs://");
}
}
public class MyFSCarbonFile extends HDFSCarbonFile {
public MyFSCarbonFile(String filePath, Configuration hadoopConf){
super(filePath,hadoopConf);
}
}
when use carbondata insert data to table, that's all right. But when select the table, i cann't scan. Because of i found to only use FileReaderImpl to read custom file.
FileFactory.java
public static FileReader getFileHolder(FileType fileType) {
return getFileHolder(fileType, getConfiguration());
}
public static FileReader getFileHolder(FileFactory.FileType fileType,
Configuration configuration) {
switch (fileType) {
case LOCAL:
return new FileReaderImpl();
case HDFS:
case ALLUXIO:
case VIEWFS:
case S3:
return new DFSFileReaderImpl(configuration);
default:
return new FileReaderImpl();
}
}
when fileType is custom，getFileHolder return is FileReaderImpl. But myfs like hdfs, it should read by DFSFileReaderImpl. Why only can use FileReaderImpl to read custom file?
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
