[GitHub] [carbondata] ravipesala commented on a change in pull request #3295: [CARBONDATA-3373] Fixed measure column filter perf

[GitBox]

ravipesala commented on a change in pull request #3295: [CARBONDATA-3373] Fixed measure column filter perf
URL: https://github.com/apache/carbondata/pull/3295#discussion_r295148136
 
 

 ##########
 File path: core/src/main/java/org/apache/carbondata/core/scan/filter/executer/MeasureColumnExecuterFilterInfo.java
 ##########
 @@ -16,15 +16,143 @@
  */
 package org.apache.carbondata.core.scan.filter.executer;
 
+import java.util.HashSet;
+import java.util.Set;
+
+import org.apache.carbondata.core.metadata.datatype.DataType;
+import org.apache.carbondata.core.metadata.datatype.DataTypes;
+
+import it.unimi.dsi.fastutil.booleans.BooleanOpenHashSet;
+import it.unimi.dsi.fastutil.bytes.ByteOpenHashSet;
+import it.unimi.dsi.fastutil.doubles.DoubleOpenHashSet;
+import it.unimi.dsi.fastutil.floats.FloatOpenHashSet;
+import it.unimi.dsi.fastutil.ints.IntOpenHashSet;
+import it.unimi.dsi.fastutil.longs.LongOpenHashSet;
+import it.unimi.dsi.fastutil.shorts.ShortOpenHashSet;
+
+/**
+ * Below class will be used to keep all the filter values based on data type
+ * for measure column.
+ * In this class there are multiple type of set is used to avoid conversion of
+ * primitive type to primitive object to avoid gc which cause perofrmace degrade when
+ * number of records are high
+ */
 public class MeasureColumnExecuterFilterInfo {
 
   Object[] filterKeys;
 
-  public void setFilterKeys(Object[] filterKeys) {
+  private ByteOpenHashSet byteOpenHashSet;
+
+  private IntOpenHashSet intOpenHashSet;
+
+  private DoubleOpenHashSet doubleOpenHashSet;
+
+  private ShortOpenHashSet shortOpenHashSet;
+
+  private Set<Object> bigDecimalHashSet;
+
+  private LongOpenHashSet longOpenHashSet;
+
+  private BooleanOpenHashSet booleanOpenHashSet;
+
+  private FloatOpenHashSet floatOpenHashSet;
 
 Review comment:
   I feel this many fileds are not necessary, just take one field with type of super class/interface. During reading you can typecast it.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
[hidden email]


With regards,
Apache Git Services