Posted by
xuchuanyin on
Sep 25, 2018; 8:53am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Issue-Bloomfilter-datamap-tp63254p63365.html
More details about this issue. I've add some logs in
`BloomCoarseGrainDataMap.createQueryModel` to print the input parameter
'expression'.
# Before applying PR2665
```
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@3b035d0c
XU expression statement:(((((((category_id <> null and device_code <> null)
and date <> null) and country_code <> null) and date >= 1472688000000000
between date <= 1472860800000000) and true) and device_code = ios-phone) and
country_code = EE)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@304f4888
XU expression statement:((((((category_id <> null and device_code <> null)
and date <> null) and country_code <> null) and date >= 1472688000000000
between date <= 1472860800000000) and true) and device_code = ios-phone)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@35b97c69
XU expression statement:(((((category_id <> null and device_code <> null)
and date <> null) and country_code <> null) and date >= 1472688000000000
between date <= 1472860800000000) and true)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@2c07277f
XU expression statement:((((category_id <> null and device_code <> null)
and date <> null) and country_code <> null) and date >= 1472688000000000
between date <= 1472860800000000)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@d4df4ce
XU expression statement:(((category_id <> null and device_code <> null) and
date <> null) and country_code <> null)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@470ce6e7
XU expression statement:((category_id <> null and device_code <> null) and
date <> null)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.logical.AndExpression@39a8905b
XU expression statement:(category_id <> null and device_code <> null)
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.conditional.NotEqualsExpression@2c8174ce
XU expression statement:category_id <> null
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@4e881e14
XU expression statement:category_id
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@6e13e2fc
XU expression statement:null
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.conditional.NotEqualsExpression@13448d2d
XU expression statement:device_code <> null
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@3444ac8f
XU expression statement:device_code
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@3ab26cad
XU expression statement:null
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.conditional.NotEqualsExpression@4b477d05
XU expression statement:date <> null
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@7c5dbca5
XU expression statement:date
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@6d6c4775
XU expression statement:null
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.conditional.NotEqualsExpression@44929971
XU expression statement:country_code <> null
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@344d6bb3
XU expression statement:country_code
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@2564410b
XU expression statement:null
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.logical.RangeExpression@2a3ced3d
XU expression statement:date >= 1472688000000000 between date <=
1472860800000000
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.conditional.GreaterThanEqualToExpression@7ab5b01a
XU expression statement:date >= 1472688000000000
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@25fa5c0c
XU expression statement:date
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@22112da1
XU expression statement:1472688000000000
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.conditional.LessThanEqualToExpression@6f0969db
XU expression statement:date <= 1472860800000000
XU expression children size:2
XU expression:
org.apache.carbondata.core.scan.expression.ColumnExpression@38eb2140
XU expression statement:date
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.LiteralExpression@4f06006d
XU expression statement:1472860800000000
XU expression children size:0
XU expression:
org.apache.carbondata.core.scan.expression.logical.TrueExpression@64514009
XU expression statement:true
XU expression children size:2
XU expression: null ------ **which causes the problem**
```
# After applying PR2665
```
XU expression:
org.apache.carbondata.core.scan.expression.conditional.EqualToExpression@4cb42a5a
XU expression statement:device_code = ios-phone
XU expression children size:2
Read 2 bloom indices from
D:/01_workspace/carbondata2/integration/spark-common/target/warehouse/carbon_bloom/bloom_dm/0/mergeShard\device_code.bloomindexmerge
XU expression:
org.apache.carbondata.core.scan.expression.conditional.EqualToExpression@6dfe4788
XU expression statement:country_code = EE
XU expression children size:2
Read 2 bloom indices from
D:/01_workspace/carbondata2/integration/spark-common/target/warehouse/carbon_bloom/bloom_dm/0/mergeShard\country_code.bloomindexmerge
XU expression:
org.apache.carbondata.core.scan.expression.conditional.EqualToExpression@4095ebff
XU expression statement:category_id = 100021
XU expression children size:2
Read 2 bloom indices from
D:/01_workspace/carbondata2/integration/spark-common/target/warehouse/carbon_bloom/bloom_dm/0/mergeShard\category_id.bloomindexmerge
XU expression:
org.apache.carbondata.core.scan.expression.conditional.InExpression@1def2d16
XU expression statement:product_id in
(LiteralExpression(590416158);LiteralExpression(590437560);)
XU expression children size:
```
We can see only bloomfilter supported expression has been forwarded.
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/