kumarvishal09 commented on a change in pull request #3438: [CARBONDATA-3531]Support load and query for MV timeseries and support multiple granularity.
URL:
https://github.com/apache/carbondata/pull/3438#discussion_r347268625
##########
File path: datamap/mv/core/src/main/scala/org/apache/carbondata/mv/datamap/MVAnalyzerRule.scala
##########
@@ -98,6 +100,50 @@ class MVAnalyzerRule(sparkSession: SparkSession) extends Rule[LogicalPlan] {
}
}
+ /**
+ * This method is specially handled for timeseries on MV, because when we use timeseries UDF which
+ * is a scala UDF, so after plan matching when query is made. We get as below query for example
+ *
+ * SELECT gen_subsumer_0.`UDF:timeseries(projectjoindate, hour)` AS `UDF:timeseries(projectjoi...
+ * FROM
+ * (SELECT datamap1_table.`UDF:timeseries_projectjoindate_hour` AS `UDF:timeseries(projectjoin...
+ * FROM
+ * default.datamap1_table
+ * GROUP BY datamap1_table.`UDF:timeseries_projectjoindate_hour`) gen_subsumer_0
+ * WHERE
+ * (UDF:timeseries(projectjoindate, hour) = TIMESTAMP('2016-02-23 09:00:00.0'))
+ *
+ * Here for Where filter expression is of type ScalaUDF, so when we do .sql() to prepare SQL, we
+ * get without qualifier name(Refer org.apache.spark.sql.catalyst.expressions.NonSQLExpression)
+ * which is 'gen_subsumer_0', so this funtion rewrites with qualifier name and returns, so that
+ * parsing does not fail in spark, for rewritten MV query.
+ * @param plan Modular Plan
+ * @param compactSQL compactSQL generated from Modular plan
+ * @return Rewritten plan with the qualifier names for where clauses in query.
+ */
+ private def reWriteTheUDFInSQLWithQualifierName(plan: ModularPlan, compactSQL: String): String = {
+ var outPutUDFColumn = ""
+ plan match {
+ case select: Select =>
+ select.outputList.collect {
+ case a: Alias if a.child.isInstanceOf[Attribute] =>
+ val childName = a.child.asInstanceOf[Attribute].name
+ if (childName.startsWith("UDF:timeseries")) {
+ outPutUDFColumn = childName
+ }
+ }
+ val queryArray = compactSQL.split("\n")
Review comment:
this split can be move down inside if check ...and in if condition we can just check whether compactSql contains WHERE clause or not. because if it contains WHERE then only split the sql
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[hidden email]
With regards,
Apache Git Services
Locked
