[GitHub] carbondata pull request #2685: [WIP] Support backward compatability in filef...

[qiuchenjian-2]

Github user manishgupta88 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2685#discussion_r215566911
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockDataMap.java ---
    @@ -631,6 +638,24 @@ private boolean useMinMaxForExecutorPruning(FilterResolverIntf filterResolverInt
         return useMinMaxForPruning;
       }
     
    +  @Override
    +  public List<Blocklet> prune(Expression expression, SegmentProperties segmentProperties,
    +      List<PartitionSpec> partitions, AbsoluteTableIdentifier identifier) throws IOException {
    +    FilterResolverIntf filterResolverIntf = null;
    +    if (expression != null) {
    +      SegmentProperties properties = getSegmentProperties();
    --- End diff --
   
    use the `segmentProperties` object getting passed in method arguments


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2685#discussion_r215855721
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockDataMap.java ---
    @@ -631,6 +638,24 @@ private boolean useMinMaxForExecutorPruning(FilterResolverIntf filterResolverInt
         return useMinMaxForPruning;
       }
     
    +  @Override
    +  public List<Blocklet> prune(Expression expression, SegmentProperties segmentProperties,
    +      List<PartitionSpec> partitions, AbsoluteTableIdentifier identifier) throws IOException {
    +    FilterResolverIntf filterResolverIntf = null;
    +    if (expression != null) {
    +      SegmentProperties properties = getSegmentProperties();
    --- End diff --
   
    ok


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/2685#discussion_r215857396
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockDataMap.java ---
    @@ -630,6 +638,24 @@ private boolean useMinMaxForExecutorPruning(FilterResolverIntf filterResolverInt
         return useMinMaxForPruning;
       }
     
    +  @Override
    +  public List<Blocklet> prune(Expression expression, SegmentProperties segmentProperties,
    +      List<PartitionSpec> partitions, AbsoluteTableIdentifier identifier) throws IOException {
    +    FilterResolverIntf filterResolverIntf = null;
    +    if (expression != null) {
    +      SegmentProperties properties = getSegmentProperties();
    +      QueryModel.FilterProcessVO processVO =
    +          new QueryModel.FilterProcessVO(properties.getDimensions(), properties.getMeasures(),
    +              new ArrayList<CarbonDimension>());
    +      QueryModel.processFilterExpression(processVO, expression, null, null);
    +      // Optimize Filter Expression and fit RANGE filters is conditions apply.
    +      FilterOptimizer rangeFilterOptimizer = new RangeFilterOptmizer(expression);
    +      rangeFilterOptimizer.optimizeFilter();
    +      filterResolverIntf = CarbonTable.resolveFilter(expression, identifier);
    --- End diff --
   
    I don't get your point. Here we need to convert to filterResolverIntf from expression for every datamap as columns meta could be different for each carbonindex.


---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    @xuchuanyin This fix is not considered for otherdatamaps as we target only for non transactional table.
    And also we dctionary_include cannot be done on non transactional table


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/134/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/302/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8372/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/305/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8377/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/139/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/149/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/317/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    Build Failed  with Spark 2.3.1, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.3/8388/



---

[qiuchenjian-2]

Github user KanakaKumar commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    LGTM


---

[qiuchenjian-2]

Github user manishgupta88 commented on the issue:

    https://github.com/apache/carbondata/pull/2685
 
    LGTM


---

[qiuchenjian-2]

Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/2685


---