One issue:
I modified the spark.version in pom.xml,using spark1.6.2, then compliation failed. Root cause: There was a "unused import statement" warinng in CarbonOptimizer class before, we imported AggregationExpression like the following : import org.apache.spark.sql.catalyst.expressions.aggregate._ import org.apache.spark.sql.catalyst.expressions._ But in spark1.6.2, AggregateExpressions is moved to subpackage "aggregate" from "expressions"(that is for spark1.5.2). So if we didn't known this change, we removed this import "import org.apache.spark.sql.catalyst.expressions.aggregate._", it will cause compliation failure when using spark1.6.2 Question: So, maybe the CI should verify carbondata based on different version spark, then it will be more helful to check the correctness of the commits, shall we? |
Administrator
|
Yes, need to solve it , the CI should support different spark version. Regards Liang
|
Yes, I Agree. CI should be configured to build Carbon on different spark
versions. On Fri, Oct 14, 2016 at 7:56 AM, Liang Chen <[hidden email]> wrote: > > Yes, need to solve it , the CI should support different spark version. > > Regards > Liang > > > zhujin wrote > > One issue: > > I modified the spark.version in pom.xml,using spark1.6.2, then > compliation > > failed. > > > > > > Root cause: > > There was a "unused import statement" warinng in CarbonOptimizer class > > before, we imported AggregationExpression like the following : > > import org.apache.spark.sql.catalyst.expressions.aggregate._ > > import org.apache.spark.sql.catalyst.expressions._ > > But in spark1.6.2, AggregateExpressions is moved to subpackage > "aggregate" > > from "expressions"(that is for spark1.5.2). > > So if we didn't known this change, we removed this import "import > > org.apache.spark.sql.catalyst.expressions.aggregate._", it will cause > > compliation failure when using spark1.6.2 > > > > > > Question: > > So, maybe the CI should verify carbondata based on different version > > spark, then it will be more helful to check the correctness of the > > commits, shall we? > > > > > > -- > View this message in context: http://apache-carbondata- > mailing-list-archive.1130556.n5.nabble.com/Disscusion- > shall-CI-support-run-carbondata-based-on-multi- > version-spark-tp1836p1890.html > Sent from the Apache CarbonData Mailing List archive mailing list archive > at Nabble.com. > |
Free forum by Nabble | Edit this page |