[GitHub] [carbondata] Indhumathi27 opened a new pull request #3541: [WIP]Timeseries query is not hitting datamap if granularity in query is given case insensitive

[GitBox]

akashrn5 commented on a change in pull request #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#discussion_r373319428
 
 

 ##########
 File path: datamap/mv/core/src/main/scala/org/apache/carbondata/mv/rewrite/Utils.scala
 ##########
 @@ -519,4 +521,73 @@ object Utils extends PredicateHelper {
     }
   }
 
+  /**
+   * Check's if timeseries udf function exists. If exists, compare literal with case insensitive
+   * value
+   */
+  def isExpressionMatchesUDF(exp: Expression, exprList: Seq[Expression]): Boolean = {
+    exp match {
+      case Alias(s: ScalaUDF, _) if s.function.isInstanceOf[TimeSeriesFunction] =>
+        val children = s.children
+        val finalExpr = exprList.filter(a1 =>
+          a1.isInstanceOf[Alias] && a1.asInstanceOf[Alias].child.isInstanceOf[ScalaUDF] &&
+          a1.asInstanceOf[Alias].child.asInstanceOf[ScalaUDF].function.
+            isInstanceOf[TimeSeriesFunction])
+        finalExpr.exists(f => {
+          val finalExp = f.asInstanceOf[Alias].child.asInstanceOf[ScalaUDF].children
+          finalExp.head.semanticEquals(children.head) &&
+          finalExp.last.asInstanceOf[Literal].toString().equalsIgnoreCase(
+            children.last.asInstanceOf[Literal].toString())
+        })
+      case s: ScalaUDF if s.function.isInstanceOf[TimeSeriesFunction] =>
+        val children = s.children
+        var expList: Seq[Expression] = Seq.empty
+        exprList foreach {
+          case s: ScalaUDF if s.function.isInstanceOf[TimeSeriesFunction] =>
+            expList = expList.+:(s)
+          case Alias(s: ScalaUDF, _) if s.function.isInstanceOf[TimeSeriesFunction] =>
+            expList = expList.+:(s.asInstanceOf[Expression])
+          case _ =>
+        }
+        expList.exists(f => {
+          val finalExp = f.asInstanceOf[ScalaUDF].children
+          finalExp.head.semanticEquals(children.head) &&
+          finalExp.last.asInstanceOf[Literal].toString().equalsIgnoreCase(
+            children.last.asInstanceOf[Literal].toString())
+        })
+      case exp: Expression =>
+        val newExp = exp.transform {
 
 Review comment:
   same as above

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#discussion_r373319949
 
 

 ##########
 File path: datamap/mv/core/src/main/scala/org/apache/carbondata/mv/rewrite/Utils.scala
 ##########
 @@ -519,4 +521,73 @@ object Utils extends PredicateHelper {
     }
   }
 
+  /**
 
 Review comment:
   Here better to add a comment, which explains the structure of the expression tree, first write the expression and better to draw a tree, of ScalaUDFExpr, children as TimeSeriesFun and Literal , so it will hep to understand the below logic in a better way.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on a change in pull request #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#discussion_r373320221
 
 

 ##########
 File path: datamap/mv/core/src/test/scala/org/apache/carbondata/mv/timeseries/TestMVTimeSeriesCreateDataMapCommand.scala
 ##########
 @@ -210,6 +210,27 @@ class TestMVTimeSeriesCreateDataMapCommand extends QueryTest with BeforeAndAfter
     }.getMessage.contains("MV Timeseries is only supported on Timestamp/Date column")
   }
 
+  test("test timeseries with case sensitive granularity") {
+    sql("drop datamap if exists datamap1")
+    sql("create datamap datamap1 on table maintable using 'mv'" +
+      " as select timeseries(projectjoindate,'Second'), sum(projectcode) from maintable group by timeseries(projectjoindate,'Second')")
+    val df1 = sql("select timeseries(projectjoindate,'SECOND'), sum(projectcode) from maintable group by timeseries(projectjoindate,'SECOND')")
+    val df2 = sql("select timeseries(projectjoinDATE,'SECOnd'), sum(projectcode) from maintable where projectcode=8 group by timeseries(projectjoinDATE,'SECOnd')")
+    TestUtil.verifyMVDataMap(df1.queryExecution.analyzed, "datamap1")
+    TestUtil.verifyMVDataMap(df2.queryExecution.analyzed, "datamap1")
+    sql("drop datamap if exists datamap1")
 
 Review comment:
   no need to again drop and create datamap, you can use the same with other query of minute level  granularity

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#issuecomment-580681844
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1816/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#issuecomment-581164426
 
 
   retest this please

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

CarbonDataQA1 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#issuecomment-581171626
 
 
   Build Success with Spark 2.3.4, Please check CI http://121.244.95.60:12545/job/ApacheCarbonPRBuilder2.3/1826/
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

akashrn5 commented on issue #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541#issuecomment-581238322
 
 
   LGTM

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services

[GitBox]

asfgit closed pull request #3541: [CARBONDATA-3636]Timeseries query is not hitting datamap if granularity in query is given case insensitive
URL: https://github.com/apache/carbondata/pull/3541
 
 
   

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services