[
https://issues.apache.org/jira/browse/CARBONDATA-1213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ravindra Pesala resolved CARBONDATA-1213.
-----------------------------------------
Resolution: Fixed
> Removed rowCountPercentage check and fixed IUD data load issue
> --------------------------------------------------------------
>
> Key: CARBONDATA-1213
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-1213> Project: CarbonData
> Issue Type: Bug
> Reporter: Manish Gupta
> Assignee: Manish Gupta
> Fix For: 1.2.0
>
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> Problems:
> 1. Row count percentage not required with high cardinality threshold check
> 2. IUD returning incorrect results in case of update on high cardinality column
> Analysis:
> 1. In case a column is identified as high cardinality column still it is not getting converted to no dictionary column because of another parameter check called rowCountPercentage. Default value of rowCountPercentage is 80%. Due to this even though high cardinality column is identified, if it is less than 80% of the total number of rows it will be treated as dictionary column. This can still lead to executor lost failure due to memory constraints.
> 2. RLE on a column is not being set correctly and due to incorrect code design RLE applicable on a column is decided by a different part of code from the one which is actually applying the RLE on a column. Because of this Footer is getting filled with incorrect RLE information and query is failing.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Locked
