[jira] [Created] (CARBONDATA-2920) For the Long string data, the local dictionary threshold is not reached even if the threshold condition is met

[Akash R Nilugal (Jira)]

Prasanna Ravichandran created CARBONDATA-2920:
-------------------------------------------------

             Summary: For the Long string data, the local dictionary threshold is not reached even if the threshold condition is met
                 Key: CARBONDATA-2920
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2920
             Project: CarbonData
          Issue Type: Bug
         Environment: 3 node cluster
            Reporter: Prasanna Ravichandran


For the Long string data, the local dictionary threshold is not reached even if the threshold condition is met.

【Test step】：
1. Create table with long string column with local dictionary threshold as 1000.
2. Load more than 1000 distinct LONG data.
3. Check if the threshold is met.

*Test queries:*

drop table if exists 1klongdata;
create table 1klongdata(st string) stored by 'carbondata' TBLPROPERTIES('local_dictionary_enable'='true','local_dictionary_threshold'='1000','long_string_columns'='st');
load data inpath "hdfs://hacluster/user/prasanna/1005longdata.csv" into table 1klongdata options('fileheader'='st');


【Expected Output】：Once the local dictionary threshold is crossed, it should display as "Local Dictionary threshold reached for the column: col_name, Unable to generate dictionary value. Dictionary threshold reached" in executor log.
【Actual Output】：It is not printing the fallback details for long data even if the threshold limit is reached.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)