Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2649#discussion_r213557758 --- Diff: core/src/main/java/org/apache/carbondata/core/metadata/schema/table/TableSchemaBuilder.java --- @@ -195,7 +196,8 @@ private ColumnSchema addColumn(StructField field, String parentName, AtomicInteg newColumn.setColumnReferenceId(newColumn.getColumnUniqueId()); newColumn.setEncodingList(createEncoding(field.getDataType(), isSortColumn, isComplexChild)); if (field.getDataType().isComplexType()) { - if (field.getDataType().getName().equalsIgnoreCase("ARRAY")) { + if (field.getDataType().getName().equalsIgnoreCase("ARRAY") || field.getDataType().getName() --- End diff -- Please use `DataTypes.isArrayType(field.getDataType())` --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2649#discussion_r213560099 --- Diff: core/src/main/java/org/apache/carbondata/core/scan/complextypes/MapQueryType.java --- @@ -0,0 +1,47 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.carbondata.core.scan.complextypes; + +import java.nio.ByteBuffer; + +import org.apache.carbondata.core.util.DataTypeUtil; + +/** + * handles querying map data type columns + */ +public class MapQueryType extends ArrayQueryType { + + public MapQueryType(String name, String parentname, int blockIndex) { + super(name, parentname, blockIndex); + } + + @Override public Object getDataBasedOnDataType(ByteBuffer dataBuffer) { --- End diff -- Please add comment to tell how we convert array to map --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2649#discussion_r213566979 --- Diff: store/sdk/src/main/java/org/apache/carbondata/sdk/file/AvroCarbonWriter.java --- @@ -134,6 +139,38 @@ private Object avroFieldToObject(Schema.Field avroField, Object fieldValue) { // also carbon internally needs float as double out = Double.parseDouble(fieldValue.toString()); break; + case MAP: + // Note: Avro object takes care of removing the duplicates so we should not handle it again + // Map will be internally stored as Array<Struct<Key,Value>> + Map mapEntries = (HashMap) fieldValue; + Object[] arrayMapChildObjects = new Object[mapEntries.size()]; + if (!mapEntries.isEmpty()) { + Iterator iterator = ((HashMap) fieldValue).entrySet().iterator(); --- End diff -- Use `mapEntries` directly instead of typecast again --- |
In reply to this post by qiuchenjian-2
Github user manishgupta88 commented on the issue:
https://github.com/apache/carbondata/pull/2649 @ravipesala ...fixed review comment. Kindly review and merge --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2649 SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/6451/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2649 Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/66/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2649 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/8137/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2649 LGTM --- |
In reply to this post by qiuchenjian-2
|
Free forum by Nabble | Edit this page |