[jira] [Updated] (CARBONDATA-2368) Support run SQL on carbon files directly, which has schema and metadata

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

xubo245 updated CARBONDATA-2368:
--------------------------------
    Description:
Now Spark DataSource changed. Carbondata fileformat class extend FileFormat class. But can not obtain files in sub-directory. Because spark filter the  files.  Any one can give idea to solve it?

for example :


{code:java}
class SparkCarbonFileFormat extends FileFormat
  with DataSourceRegister
  with Logging
  with Serializable {

  override def inferSchema(sparkSession: SparkSession,
      options: Map[String, String],
      files: Seq[FileStatus]): Option[StructType] = {
...
{code}


  was:
Now Spark DataSource changed. Carbondata fileformat class extend FileFormat class. But can not obtain files in sub-directory. Any one can give idea to solve it?

for example :


{code:java}
class SparkCarbonFileFormat extends FileFormat
  with DataSourceRegister
  with Logging
  with Serializable {

  override def inferSchema(sparkSession: SparkSession,
      options: Map[String, String],
      files: Seq[FileStatus]): Option[StructType] = {
...
{code}

> Support run SQL on carbon files directly, which has schema and metadata
> -----------------------------------------------------------------------
>
>                 Key: CARBONDATA-2368
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2368
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: xubo245
>            Assignee: xubo245
>            Priority: Major
>
> Now Spark DataSource changed. Carbondata fileformat class extend FileFormat class. But can not obtain files in sub-directory. Because spark filter the  files.  Any one can give idea to solve it?
> for example :
> {code:java}
> class SparkCarbonFileFormat extends FileFormat
>   with DataSourceRegister
>   with Logging
>   with Serializable {
>   override def inferSchema(sparkSession: SparkSession,
>       options: Map[String, String],
>       files: Seq[FileStatus]): Option[StructType] = {
> ...
> {code}

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)