[jira] [Resolved] (CARBONDATA-3036) Carbon 1.5.0 B010 - Select query fails when min/max exceeds and index tree cached

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor resolved CARBONDATA-3036.
--------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.5.1

> Carbon 1.5.0 B010 - Select query fails when min/max exceeds and index tree cached
> ---------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-3036
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3036
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-query
>    Affects Versions: 1.5.0
>         Environment: 17 node cluster
> 0: jdbc:hive2://192.168.137.14:22550/default> show segments for table jinling_localsort_3;
> +--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+
> | SegmentSequenceId  |  Status  |     Load Start Time      |      Load End Time       | Merged To  | File Format  | Data Size  | Index Size  |
> +--------------------+----------+--------------------------+--------------------------+------------+--------------+------------+-------------+--+
> | 8                  | Success  | 2018-10-17 08:30:23.103  | 2018-10-17 08:30:28.476  | NA         | COLUMNAR_V3  | 185.68MB   | 243.36KB    |
> | 7                  | Success  | 2018-10-17 08:30:16.359  | 2018-10-17 08:30:21.667  | NA         | COLUMNAR_V3  | 185.68MB   | 243.36KB    |
> | 6                  | Success  | 2018-10-17 08:30:09.439  | 2018-10-17 08:30:14.97   | NA         | COLUMNAR_V3  | 185.68MB   | 243.36KB    |
> | 5                  | Success  | 2018-10-17 08:30:02.688  | 2018-10-17 08:30:08.036  | NA         | COLUMNAR_V3  | 186.02MB   | 243.35KB    |
> | 4                  | Success  | 2018-10-17 08:27:09.09   | 2018-10-17 08:27:14.553  | NA         | COLUMNAR_V3  | 186.34MB   | 243.35KB    |
> | 3                  | Success  | 2018-10-17 08:27:00.659  | 2018-10-17 08:27:06.096  | NA         | COLUMNAR_V3  | 185.68MB   | 243.36KB    |
> | 2                  | Success  | 2018-10-17 08:26:52.047  | 2018-10-17 08:26:57.47   | NA         | COLUMNAR_V3  | 186.02MB   | 243.35KB    |
> | 1                  | Success  | 2018-10-17 08:26:43.462  | 2018-10-17 08:26:48.899  | NA         | COLUMNAR_V3  | 186.34MB   | 243.35KB    |
> | 0                  | Success  | 2018-10-17 08:20:46.654  | 2018-10-17 08:20:52.681  | NA         | COLUMNAR_V3  | 186.02MB   | 243.35KB    |
>            Reporter: Ramakrishna S
>            Assignee: MANISH NALLA
>            Priority: Major
>             Fix For: 1.5.1
>
>          Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> If configured cache columns and min, max present flag columns is different*, it is possible to get wrong data* due to wrong index is mapped.
> results inconsistent when cache is set but min/max exceeds. Column is dictionary excluded.
> set carbon.minmax.allowed.byte.count=50;
>  
> 0: jdbc:hive2://192.168.137.14:22550/default> use db01;
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (0.06 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> show tables;
> +-----------+----------------------+--------------+--+
> | database | tableName | isTemporary |
> +-----------+----------------------+--------------+--+
> | db01 | jinling_localsort_3 | false |
> +-----------+----------------------+--------------+--+
> 1 row selected (0.068 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> alter table jinling_localsort_3 set TBLPROPERTIES('COLUMN_META_CACHE'='user_imsi,user_num,dim24');
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (3.495 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> select count(*) from jinling_localsort_3;
> +-----------+--+
> | count(1) |
> +-----------+--+
> | 9000000 |
> +-----------+--+
> 1 row selected (2.606 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> select dim24 from jinling_localsort_3 limit 2;
> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
> | dim24 |
> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
> | E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA |
> | Q4Plek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA |
> +------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
> 2 rows selected (457.462 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> select count(*) from jinling_localsort_3 where dim24 = 'E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA';
> +-----------+--+
> | count(1) |
> +-----------+--+
> | 0 |
> +-----------+--+
> 1 row selected (0.871 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> alter table jinling_localsort_3 set TBLPROPERTIES('COLUMN_META_CACHE'='user_imsi,user_num');
> +---------+--+
> | Result |
> +---------+--+
> +---------+--+
> No rows selected (3.798 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> select count(*) from jinling_localsort_3;
> +-----------+--+
> | count(1) |
> +-----------+--+
> | 9000000 |
> +-----------+--+
> 1 row selected (4.162 seconds)
> 0: jdbc:hive2://192.168.137.14:22550/default> select count(*) from jinling_localsort_3 where dim24 = 'E6Phek Hobbit from the Shire and eight companions set out on a journey to destroy the powerful One Ring and save Middle-earth from the Dark Lord Sauron.AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA';
> +-----------+--+
> | count(1) |
> +-----------+--+
> | 90 |
> +-----------+--+
> 1 row selected (4.515 seconds)
>  

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)