[jira] [Commented] (CARBONDATA-285) Use path parameter in Spark datasource API

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CARBONDATA-285) Use path parameter in Spark datasource API

Akash R Nilugal (Jira)

    [ https://issues.apache.org/jira/browse/CARBONDATA-285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15573945#comment-15573945 ]

ASF GitHub Bot commented on CARBONDATA-285:
-------------------------------------------

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/212#discussion_r83350430
 
    --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/execution/command/carbonTableSchema.scala ---
    @@ -861,9 +861,11 @@ private[sql] case class CreateTable(cm: tableModel) extends RunnableCommand {
           val tablePath = catalog.createTableFromThrift(tableInfo, dbName, tbName, null)(sqlContext)
           try {
             sqlContext.sql(
    -          s"""CREATE TABLE $dbName.$tbName USING carbondata""" +
    -          s""" OPTIONS (tableName "$dbName.$tbName", tablePath "$tablePath") """)
    -              .collect
    +          s"""
    +             | CREATE TABLE $dbName.$tbName
    +             | USING carbondata
    +             | OPTIONS (path "$tablePath")
    --- End diff --
   
    There would be backward compatability issues here. Old tables cannot work because `path` was not present.


> Use path parameter in Spark datasource API
> ------------------------------------------
>
>                 Key: CARBONDATA-285
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-285
>             Project: CarbonData
>          Issue Type: Improvement
>          Components: spark-integration
>    Affects Versions: 0.1.0-incubating
>            Reporter: Jacky Li
>             Fix For: 0.2.0-incubating
>
>
> Currently, when using carbon with spark datasource API, it need to give database name and table name as parameter, it is not the normal way of datasource API usage. In this PR, database name and table name is not required to give, user need to specify the `path` parameter (indicating the path to table folder) only when using datasource API



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)