Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1458/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1875/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 @jackylk Please review, I have moved the `like` expression conversion to carbon layer in this PR itself, so like expression can be accessed as `STARTSWITH` or `ENDSWITH` or `CONTAINSWITH` to the datamap --- |
In reply to this post by qiuchenjian-2
Github user xubochin commented on the issue:
https://github.com/apache/carbondata/pull/1510 Thought this code , I understand that : STARTSWITH is "%xxx",ENDSWITH is "%xxx",CONTAINSWITH is "xxx", is it right ? Luceen only use STARTSWITH. --- |
In reply to this post by qiuchenjian-2
Github user xubochin commented on the issue:
https://github.com/apache/carbondata/pull/1510 except ENDSWITH , other like string can pass to Lucene ? --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 @xubochin LuceneDataMap can only include the expression type which it supports inside metadata so that remaining expressions carbon can handle it. --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/1510#discussion_r156602474 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/DataMapChooser.java --- @@ -0,0 +1,269 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.carbondata.core.datamap; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashSet; +import java.util.List; +import java.util.Set; + +import org.apache.carbondata.core.datamap.dev.expr.AndDataMapExprWrapper; +import org.apache.carbondata.core.datamap.dev.expr.DataMapExprWrapper; +import org.apache.carbondata.core.datamap.dev.expr.DataMapExprWrapperImpl; +import org.apache.carbondata.core.datamap.dev.expr.OrDataMapExprWrapper; +import org.apache.carbondata.core.metadata.schema.table.CarbonTable; +import org.apache.carbondata.core.scan.expression.ColumnExpression; +import org.apache.carbondata.core.scan.expression.Expression; +import org.apache.carbondata.core.scan.expression.logical.AndExpression; +import org.apache.carbondata.core.scan.expression.logical.OrExpression; +import org.apache.carbondata.core.scan.filter.intf.ExpressionType; +import org.apache.carbondata.core.scan.filter.resolver.FilterResolverIntf; +import org.apache.carbondata.core.scan.filter.resolver.resolverinfo.TrueConditionalResolverImpl; + +/** + * This chooser does 2 jobs. + * 1. Based on filter expression it converts the available datamaps to datamap expression. + * For example, there are 2 datamaps available on table1 + * Datamap1 : column1 + * Datamap2 : column2 + * Query: select * from table1 where column1 ='a' and column2 =b + * For the above query, we create datamap expression as AndDataMapExpression(Datamap1, DataMap2). + * So for the above query both the datamaps are included and the output of them will be + * applied AND condition to improve the performance + * + * 2. It chooses the datamap out of available datamaps based on simple logic. + * Like if there is filter condition on column1 then for + * supposing 2 datamaps(1. column1 2. column1+column2) are supporting this column then we choose + * the datamap which has fewer columns that is the first datamap. + */ +public class DataMapChooser { + + public DataMapExprWrapper choose(CarbonTable carbonTable, + FilterResolverIntf resolverIntf) { + if (resolverIntf != null) { + Expression expression = resolverIntf.getFilterExpression(); + // First check for FG datamaps if any exist + List<TableDataMap> allDataMapFG = + DataMapStoreManager.getInstance().getAllDataMap(carbonTable, DataMapType.FG); + ExpressionTuple tuple = selectDataMap(expression, allDataMapFG); + if (tuple.dataMapExprWrapper == null) { + // Check for CG datamap + List<TableDataMap> allDataMapCG = + DataMapStoreManager.getInstance().getAllDataMap(carbonTable, DataMapType.CG); + tuple = selectDataMap(expression, allDataMapCG); + } + if (tuple.dataMapExprWrapper != null) { + return tuple.dataMapExprWrapper; + } + } + // Return the default datamap if no other datamap exists. + return new DataMapExprWrapperImpl(DataMapStoreManager.getInstance() --- End diff -- make the line wrap easier to read, like: ``` return new DataMapExprWrapperImpl( DataMapStoreManager.getInstance().getDefaultDataMap( carbonTable.getAbsoluteTableIdentifier()), resolverIntf); ``` --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1092/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2308/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2543/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/1953/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3152/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3187/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2284/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3521/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/1510 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/3367/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3783/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/2538/ --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on the issue:
https://github.com/apache/carbondata/pull/1510 retest this please --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/1510 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/3786/ --- |
Free forum by Nabble | Edit this page |