[jira] [Created] (CARBONDATA-2576) MV Datamap - MV is not working fine if there is more than 3 aggregate function in the same datamap.

[Akash R Nilugal (Jira)]

Prasanna Ravichandran created CARBONDATA-2576:
-------------------------------------------------

             Summary: MV Datamap - MV is not working fine if there is more than 3 aggregate function in the same datamap.
                 Key: CARBONDATA-2576
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2576
             Project: CarbonData
          Issue Type: Bug
          Components: data-query
            Reporter: Prasanna Ravichandran
         Attachments: From 4th aggregate function -error shown.docx

MV is not working fine if there is more than 3 aggregate function in the same datamap.

Test queries:

 

scala> carbon.sql("create datamap datamap_comp_maxsumminavg using 'mv' as select empno,max(projectenddate),sum(salary),min(projectjoindate),avg(attendance) from originTable group by empno").show(200,false)

++

||

++

++

 

 

scala> carbon.sql("rebuild datamap datamap_comp_maxsumminavg").show(200,false)

++

||

++

++

 

 

scala> carbon.sql("explain select empno,max(projectenddate),sum(salary),min(projectjoindate),avg(attendance) from originTable group by empno").show(200,false)

org.apache.spark.sql.AnalysisException: expression 'datamap_comp_maxsumminavg_table.`avg_attendance`' is neither present in the group by, nor is it an aggregate function. Add to group by or wrap in first() (or first_value) if you don't care which value you get.;;

Aggregate [origintable_empno#2925], [origintable_empno#2925 AS empno#3002, max(max_projectenddate#2926) AS max(projectenddate)#3003, sum(sum_salary#2927L) AS sum(salary)#3004L, min(min_projectjoindate#2928) AS min(projectjoindate)#3005, avg_attendance#2929 AS avg(attendance)#3006]

+- SubqueryAlias datamap_comp_maxsumminavg_table

   +- Relation[origintable_empno#2925,max_projectenddate#2926,sum_salary#2927L,min_projectjoindate#2928,avg_attendance#2929] CarbonDatasourceHadoopRelation [ Database name :default, Table name :datamap_comp_maxsumminavg_table, Schema :Some(StructType(StructField(origintable_empno,IntegerType,true), StructField(max_projectenddate,TimestampType,true), StructField(sum_salary,LongType,true), StructField(min_projectjoindate,TimestampType,true), StructField(avg_attendance,DoubleType,true))) ]

 

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.failAnalysis(CheckAnalysis.scala:39)

  at org.apache.spark.sql.catalyst.analysis.Analyzer.failAnalysis(Analyzer.scala:91)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.org$apache$spark$sql$catalyst$analysis$CheckAnalysis$class$$anonfun$$checkValidAggregateExpression$1(CheckAnalysis.scala:247)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$org$apache$spark$sql$catalyst$analysis$CheckAnalysis$class$$anonfun$$checkValidAggregateExpression$1$5.apply(CheckAnalysis.scala:253)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$org$apache$spark$sql$catalyst$analysis$CheckAnalysis$class$$anonfun$$checkValidAggregateExpression$1$5.apply(CheckAnalysis.scala:253)

  at scala.collection.immutable.List.foreach(List.scala:381)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.org$apache$spark$sql$catalyst$analysis$CheckAnalysis$class$$anonfun$$checkValidAggregateExpression$1(CheckAnalysis.scala:253)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$9.apply(CheckAnalysis.scala:280)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1$$anonfun$apply$9.apply(CheckAnalysis.scala:280)

  at scala.collection.immutable.List.foreach(List.scala:381)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:280)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$$anonfun$checkAnalysis$1.apply(CheckAnalysis.scala:78)

  at org.apache.spark.sql.catalyst.trees.TreeNode.foreachUp(TreeNode.scala:127)

  at org.apache.spark.sql.catalyst.analysis.CheckAnalysis$class.checkAnalysis(CheckAnalysis.scala:78)

  at org.apache.spark.sql.catalyst.analysis.Analyzer.checkAnalysis(Analyzer.scala:91)

  at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:52)

  at org.apache.spark.sql.CarbonSession.withProfiler(CarbonSession.scala:148)

  at org.apache.spark.sql.CarbonSession.sql(CarbonSession.scala:95)

  at org.apache.carbondata.mv.datamap.MVAnalyzerRule.apply(MVAnalyzerRule.scala:72)

  at org.apache.carbondata.mv.datamap.MVAnalyzerRule.apply(MVAnalyzerRule.scala:38)

  at org.apache.spark.sql.hive.CarbonAnalyzer.execute(CarbonAnalyzer.scala:46)

  at org.apache.spark.sql.hive.CarbonAnalyzer.execute(CarbonAnalyzer.scala:27)

  at org.apache.spark.sql.execution.QueryExecution.analyzed$lzycompute(QueryExecution.scala:69)

  at org.apache.spark.sql.execution.QueryExecution.analyzed(QueryExecution.scala:67)

  at org.apache.spark.sql.execution.QueryExecution.assertAnalyzed(QueryExecution.scala:50)

  at org.apache.spark.sql.execution.QueryExecution.withCachedData$lzycompute(QueryExecution.scala:73)

  at org.apache.spark.sql.execution.QueryExecution.withCachedData(QueryExecution.scala:72)

  at org.apache.spark.sql.execution.QueryExecution.optimizedPlan$lzycompute(QueryExecution.scala:78)

  at org.apache.spark.sql.execution.QueryExecution.optimizedPlan(QueryExecution.scala:78)

  at org.apache.spark.sql.execution.QueryExecution.sparkPlan$lzycompute(QueryExecution.scala:84)

  at org.apache.spark.sql.execution.QueryExecution.sparkPlan(QueryExecution.scala:80)

  at org.apache.spark.sql.execution.QueryExecution.executedPlan$lzycompute(QueryExecution.scala:89)

  at org.apache.spark.sql.execution.QueryExecution.executedPlan(QueryExecution.scala:89)

  at org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:92)

  at org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:92)

  at org.apache.spark.sql.execution.command.table.CarbonExplainCommand.collectProfiler(CarbonExplainCommand.scala:54)

  at org.apache.spark.sql.execution.command.table.CarbonExplainCommand.processMetadata(CarbonExplainCommand.scala:45)

  at org.apache.spark.sql.execution.command.MetadataCommand.run(package.scala:68)

  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult$lzycompute(commands.scala:58)

  at org.apache.spark.sql.execution.command.ExecutedCommandExec.sideEffectResult(commands.scala:56)

  at org.apache.spark.sql.execution.command.ExecutedCommandExec.executeCollect(commands.scala:67)

  at org.apache.spark.sql.Dataset.<init>(Dataset.scala:183)

  at org.apache.spark.sql.CarbonSession$$anonfun$sql$1.apply(CarbonSession.scala:108)

  at org.apache.spark.sql.CarbonSession$$anonfun$sql$1.apply(CarbonSession.scala:97)

  at org.apache.spark.sql.CarbonSession.withProfiler(CarbonSession.scala:155)

  at org.apache.spark.sql.CarbonSession.sql(CarbonSession.scala:95)

  ... 48 elided



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)