Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6636/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6638/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5466/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5504/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2411 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5534/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6679/ --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200017553 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper --- End diff -- Can you describe why this is needed? --- |
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200018301 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper --- End diff -- Because currently a DatamapWrapper contains more than one datamap, this class is to describle its schema. For example a AndDatamapWrapper contains BloomFilterin its left and Lucene in its right, then its schema would be AND(BloomFilter, Lucene). This DataMapWrapperSchema is used for explain output --- |
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200018398 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper + */ +public class DataMapWrapperSchema { + enum WrapperType { + PRIMITIVE, + AND, + OR + } + + private WrapperType wrapperType; + private DataMapWrapperSchema left; + private DataMapWrapperSchema right; + private DataMapSchema schema; + + private DataMapWrapperSchema(WrapperType wrapperType, DataMapWrapperSchema left, + DataMapWrapperSchema right) { + this.wrapperType = wrapperType; + this.left = left; + this.right = right; + } + + private DataMapWrapperSchema(DataMapSchema schema) { + this.wrapperType = WrapperType.PRIMITIVE; + this.schema = schema; + } + + public static DataMapWrapperSchema fromDataMapWrapper(DataMapExprWrapper dataMapExprWrapper) { + if (dataMapExprWrapper instanceof DataMapExprWrapperImpl) { + return new DataMapWrapperSchema( + ((DataMapExprWrapperImpl) dataMapExprWrapper).getDataMapSchema()); + } else if (dataMapExprWrapper instanceof AndDataMapExprWrapper) { + return new DataMapWrapperSchema(WrapperType.AND, + fromDataMapWrapper(dataMapExprWrapper.getLeftDataMapWrapper()), + fromDataMapWrapper(dataMapExprWrapper.getRightDataMapWrapprt())); + } else { + return new DataMapWrapperSchema(WrapperType.OR, + fromDataMapWrapper(dataMapExprWrapper.getLeftDataMapWrapper()), + fromDataMapWrapper(dataMapExprWrapper.getRightDataMapWrapprt())); + } + } + + public String getDataMapWrapperName() { + if (WrapperType.PRIMITIVE == wrapperType) { + return schema.getDataMapName(); + } else { + return String.format("%s(%s, %s)", --- End diff -- FYI: This is the wrapperName --- |
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200018440 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper + */ +public class DataMapWrapperSchema { + enum WrapperType { + PRIMITIVE, + AND, + OR + } + + private WrapperType wrapperType; + private DataMapWrapperSchema left; + private DataMapWrapperSchema right; + private DataMapSchema schema; + + private DataMapWrapperSchema(WrapperType wrapperType, DataMapWrapperSchema left, + DataMapWrapperSchema right) { + this.wrapperType = wrapperType; + this.left = left; + this.right = right; + } + + private DataMapWrapperSchema(DataMapSchema schema) { + this.wrapperType = WrapperType.PRIMITIVE; + this.schema = schema; + } + + public static DataMapWrapperSchema fromDataMapWrapper(DataMapExprWrapper dataMapExprWrapper) { + if (dataMapExprWrapper instanceof DataMapExprWrapperImpl) { + return new DataMapWrapperSchema( + ((DataMapExprWrapperImpl) dataMapExprWrapper).getDataMapSchema()); + } else if (dataMapExprWrapper instanceof AndDataMapExprWrapper) { + return new DataMapWrapperSchema(WrapperType.AND, + fromDataMapWrapper(dataMapExprWrapper.getLeftDataMapWrapper()), + fromDataMapWrapper(dataMapExprWrapper.getRightDataMapWrapprt())); + } else { + return new DataMapWrapperSchema(WrapperType.OR, + fromDataMapWrapper(dataMapExprWrapper.getLeftDataMapWrapper()), + fromDataMapWrapper(dataMapExprWrapper.getRightDataMapWrapprt())); + } + } + + public String getDataMapWrapperName() { + if (WrapperType.PRIMITIVE == wrapperType) { + return schema.getDataMapName(); + } else { + return String.format("%s(%s, %s)", + wrapperType, left.getDataMapWrapperName(), right.getDataMapWrapperName()); + } + } + + public String getDataMapWrapperProvider() { + if (WrapperType.PRIMITIVE == wrapperType) { + return schema.getProviderName(); + } else { + return String.format("%s(%s, %s)", --- End diff -- FYI: This is the wrapperProvider --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200018803 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper --- End diff -- Can you add these in comment. And if this class is only used for Explain, can you give a better name for it. --- |
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2411#discussion_r200021222 --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/expr/DataMapWrapperSchema.java --- @@ -0,0 +1,87 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.carbondata.core.datamap.dev.expr; + +import org.apache.carbondata.core.metadata.schema.table.DataMapSchema; + +/** + * schema for datamap wrapper --- End diff -- OK, will change the name to 'DataMapWrapperSimpleInfo' --- |
In reply to this post by qiuchenjian-2
Github user xuchuanyin commented on the issue:
https://github.com/apache/carbondata/pull/2411 The output for explain looks like below: ``` Table Scan on carbon_bloom - total blocklets: 1 - filter: (((((name <> null and city <> null) and id <> null) and id = 1) and city = city_1) and name = n1) - pruned by Main DataMap - skipped blocklets: 0 - pruned by CG DataMap - name: AND(AND(datamap11, datamap13), datamap12) - provider: AND(AND(bloomfilter, bloomfilter), bloomfilter) - skipped blocklets: 0 ``` Before this PR, the output only shows `datamap11` --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2411 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5595/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/6756/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5588/ --- |
In reply to this post by qiuchenjian-2
|
In reply to this post by qiuchenjian-2
|
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2411 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5611/ --- |
Free forum by Nabble | Edit this page |