[GitHub] carbondata pull request #2627: [CARBONDATA-2835] [MVDataMap] Block MV datama...

[qiuchenjian-2]

Github user ndwangsen commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2627#discussion_r210241115
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/datamap/CarbonCreateDataMapCommand.scala ---
    @@ -73,13 +73,8 @@ case class CarbonCreateDataMapCommand(
           }
         }
     
    -    if (mainTable != null &&
    -        mainTable.isStreamingSink &&
    -        !(dmProviderName.equalsIgnoreCase(DataMapClassProvider.PREAGGREGATE.toString)
    -          || dmProviderName.equalsIgnoreCase(DataMapClassProvider.TIMESERIES.toString))) {
    -      throw new MalformedCarbonCommandException(s"Streaming table does not support creating " +
    -                                                s"$dmProviderName datamap")
    -    }
    +    // delete this code because streaming table only does not support creating MV datamap,
    --- End diff --
   
    Delete the comments here


---

[qiuchenjian-2]

Github user ndwangsen commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2627#discussion_r210241324
 
    --- Diff: datamap/mv/core/src/main/scala/org/apache/carbondata/mv/datamap/MVHelper.scala ---
    @@ -80,6 +81,16 @@ object MVHelper {
         dmProperties.foreach(t => tableProperties.put(t._1, t._2))
     
         val selectTables = getTables(logicalPlan)
    +    selectTables.map { selectTable =>
    +      val mainCarbonTable = CarbonEnv.getCarbonTableOption(selectTable.identifier.database,
    +        selectTable.identifier.table)(sparkSession)
    +
    +      if (!mainCarbonTable.isEmpty && mainCarbonTable.get.isStreamingSink ) {
    +        throw new MalformedCarbonCommandException(s"Streaming table does not support creating " +
    +                                                    s"MV datamap")
    +      }
    +      selectTable
    --- End diff --
   
    ok，I remove it


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7918/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/6642/



---

[qiuchenjian-2]

Github user ndwangsen commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6275/



---

[qiuchenjian-2]

Github user ndwangsen commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6280/



---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2627#discussion_r210782975
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala ---
    @@ -237,6 +237,21 @@ object CarbonEnv {
         getCarbonTable(tableIdentifier.database, tableIdentifier.table)(sparkSession)
       }
     
    +  /**
    +   * This method returns corresponding CarbonTable, it will return None if it's not a CarbonTable
    +   */
    +  def getCarbonTableOption(
    --- End diff --
   
    There is already a `getCarbonTable` func defined in `CarbonEnv`, is this needed??


---

[qiuchenjian-2]

Github user ndwangsen commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2627#discussion_r210787370
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/CarbonEnv.scala ---
    @@ -237,6 +237,21 @@ object CarbonEnv {
         getCarbonTable(tableIdentifier.database, tableIdentifier.table)(sparkSession)
       }
     
    +  /**
    +   * This method returns corresponding CarbonTable, it will return None if it's not a CarbonTable
    +   */
    +  def getCarbonTableOption(
    --- End diff --
   
    the getCarbonTable will throw a exception when get a non-carbon table


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6454/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8141/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/70/



---

[qiuchenjian-2]

Github user ndwangsen commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    retest sdv please


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6471/



---

[qiuchenjian-2]

Github user jackylk commented on the issue:

    https://github.com/apache/carbondata/pull/2627
 
    retest sdv please


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2627


---