[GitHub] carbondata pull request #2100: [WIP] Save the datamaps to system folder of w...

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2100: [CARBONDATA-2278] Save the datamaps to system...

qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2100#discussion_r177651996
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/DataMapCatalog.java ---
    @@ -0,0 +1,51 @@
    +/*
    + * Licensed to the Apache Software Foundation (ASF) under one or more
    + * contributor license agreements.  See the NOTICE file distributed with
    + * this work for additional information regarding copyright ownership.
    + * The ASF licenses this file to You under the Apache License, Version 2.0
    + * (the "License"); you may not use this file except in compliance with
    + * the License.  You may obtain a copy of the License at
    + *
    + *    http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.carbondata.core.datamap;
    +
    +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema;
    +
    +/**
    + * This is the interface for inmemory catalog registry for datamap.
    + * @since 1.4.0
    + */
    +public interface DataMapCatalog<T> {
    --- End diff --
   
    ok


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2100: [CARBONDATA-2278] Save the datamaps to system...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2100#discussion_r177652486
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/parser/CarbonSpark2SqlParser.scala ---
    @@ -150,34 +150,39 @@ class CarbonSpark2SqlParser extends CarbonDDLSqlParser {
        */
       protected lazy val createDataMap: Parser[LogicalPlan] =
         CREATE ~> DATAMAP ~> opt(IF ~> NOT ~> EXISTS) ~ ident ~
    -    (ON ~ TABLE) ~  (ident <~ ".").? ~ ident ~
    +    opt(ontable) ~
         (USING ~> stringLit) ~ (DMPROPERTIES ~> "(" ~> repsep(loadOptions, ",") <~ ")").? ~
         (AS ~> restInput).? <~ opt(";") ^^ {
    -      case ifnotexists ~ dmname ~ ontable ~ dbName ~ tableName ~ className ~ dmprops ~ query =>
    +      case ifnotexists ~ dmname ~ tableIdent ~ className ~ dmprops ~ query =>
    +
             val map = dmprops.getOrElse(List[(String, String)]()).toMap[String, String]
    -        CarbonCreateDataMapCommand(
    -          dmname, TableIdentifier(tableName, dbName), className, map, query, ifnotexists.isDefined)
    +        CarbonCreateDataMapCommand(dmname, tableIdent, className, map, query, ifnotexists.isDefined)
    +    }
    +
    +  protected lazy val ontable: Parser[TableIdentifier] =
    +    ON ~> TABLE ~>  (ident <~ ".").? ~ ident ^^ {
    +      case dbName ~ tableName =>
    +        TableIdentifier(tableName, dbName)
         }
     
       /**
        * The below syntax is used to drop the datamap.
        * DROP DATAMAP IF EXISTS datamapName ON TABLE tablename
        */
       protected lazy val dropDataMap: Parser[LogicalPlan] =
    -    DROP ~> DATAMAP ~> opt(IF ~> EXISTS) ~ ident ~ (ON ~ TABLE) ~
    -    (ident <~ ".").? ~ ident <~ opt(";")  ^^ {
    -      case ifexists ~ dmname ~ ontable ~ dbName ~ tableName =>
    -        CarbonDropDataMapCommand(dmname, ifexists.isDefined, dbName, tableName)
    +    DROP ~> DATAMAP ~> opt(IF ~> EXISTS) ~ ident ~ opt(ontable) <~ opt(";")  ^^ {
    +      case ifexists ~ dmname ~ tableIdent =>
    +        CarbonDropDataMapCommand(dmname, ifexists.isDefined, tableIdent)
         }
     
       /**
        * The syntax of show datamap is used to show datamaps on the table
        * SHOW DATAMAP ON TABLE tableName
        */
       protected lazy val showDataMap: Parser[LogicalPlan] =
    -    SHOW ~> DATAMAP ~> ON ~> TABLE ~> (ident <~ ".").? ~ ident <~ opt(";") ^^ {
    -      case databaseName ~ tableName =>
    -        CarbonDataMapShowCommand(convertDbNameToLowerCase(databaseName), tableName.toLowerCase())
    +    SHOW ~> DATAMAP ~> opt(ontable) <~ opt(";") ^^ {
    --- End diff --
   
    it should be but I am scared about backward compatability


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2100: [CARBONDATA-2278] Save the datamaps to system folder...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2100
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/3402/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2100: [CARBONDATA-2278] Save the datamaps to system folder...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2100
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/4627/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2100: [CARBONDATA-2278] Save the datamaps to system folder...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2100
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/4134/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #2100: [CARBONDATA-2278] Save the datamaps to system folder...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/2100
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #2100: [CARBONDATA-2278] Save the datamaps to system...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2100


---
12