[jira] [Updated] (CARBONDATA-4029) After delete in the table which has Alter-added SDK segments, then the count(*) is 0.

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prasanna Ravichandran updated CARBONDATA-4029:
----------------------------------------------
    Summary: After delete in the table which has Alter-added SDK segments, then the count(*) is 0.  (was: Getting Number format exception while querying on date columns in SDK carbon table.)

> After delete in the table which has Alter-added SDK segments, then the count(*) is 0.
> -------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4029
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4029
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>         Environment: 3 node FI cluster
>            Reporter: Prasanna Ravichandran
>            Priority: Minor
>         Attachments: Primitive.rar
>
>
> We are getting Number format exception while querying on the date columns. Attached the SDK files also.
> Test queries:
> --SDK compaction;
>  drop table if exists external_primitive;
>  create table external_primitive (id int, name string, rank smallint, salary double, active boolean, dob date, doj timestamp, city string, dept string) stored as carbondata;
>  alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive','format'='carbon');
>  alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive2','format'='carbon');
>  alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive3','format'='carbon');
>  alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive4','format'='carbon');
>  alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive5','format'='carbon');
>  
>  alter table external_primitive compact 'minor'; --working fine pass;
>  select count(*) from external_primitive;--working fine pass;
> show segments for table external_primitive;
>  select * from external_primitive limit 13; --working fine pass;
>  select * from external_primitive limit 14; --failed getting number format exception;
> select min(dob) from external_primitive; --failed getting number format exception;
> select max(dob) from external_primitive; --working;
> select dob from external_primitive; --failed getting number format exception;
> Console:
> *0: /> show segments for table external_primitive;*
> +------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
> | ID | Status | Load Start Time | Load Time Taken | Partition | Data Size | Index Size | File Format |
> +------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
> | 4 | Success | 2020-10-13 11:52:04.012 | 0.511S | {} | 1.88KB | 655.0B | columnar_v3 |
> | 3 | Compacted | 2020-10-13 11:52:00.587 | 0.828S | {} | 1.88KB | 655.0B | columnar_v3 |
> | 2 | Compacted | 2020-10-13 11:51:57.767 | 0.775S | {} | 1.88KB | 655.0B | columnar_v3 |
> | 1 | Compacted | 2020-10-13 11:51:54.678 | 1.024S | {} | 1.88KB | 655.0B | columnar_v3 |
> | 0.1 | Success | 2020-10-13 11:52:05.986 | 5.785S | {} | 9.62KB | 5.01KB | columnar_v3 |
> | 0 | Compacted | 2020-10-13 11:51:51.072 | 1.125S | {} | 8.55KB | 4.25KB | columnar_v3 |
> +------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
> 6 rows selected (0.45 seconds)
> *0: /> select * from external_primitive limit 13;* --working fine pass;
> INFO : Execution ID: 95
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> | id | name | rank | salary | active | dob | doj | city | dept |
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> | 1 | AAA | 3 | 3444345.66 | true | 1979-12-09 | 2011-02-09 22:30:20.0 | Pune | IT |
> | 2 | BBB | 2 | 543124.66 | false | 1987-02-19 | 2017-01-01 09:30:20.0 | Bangalore | DATA |
> | 3 | CCC | 1 | 787878.888 | false | 1982-05-12 | 2015-11-30 23:50:20.0 | Pune | DATA |
> | 4 | DDD | 1 | 99999.24 | true | 1981-04-09 | 2000-01-15 04:30:20.0 | Delhi | MAINS |
> | 5 | EEE | 3 | 545656.99 | true | 1987-12-09 | 2017-11-25 01:30:20.0 | Delhi | IT |
> | 6 | FFF | 2 | 768678.0 | false | 1987-12-20 | 2017-01-10 02:30:20.0 | Bangalore | DATA |
> | 7 | GGG | 3 | 765665.0 | true | 1983-06-12 | 2016-12-31 23:30:20.0 | Pune | IT |
> | 8 | HHH | 2 | 567567.66 | false | 1979-01-12 | 1995-01-01 09:30:20.0 | Bangalore | DATA |
> | 9 | III | 2 | 787878.767 | true | 1985-02-19 | 2005-08-14 22:30:20.0 | Pune | DATA |
> | 10 | JJJ | 3 | 887877.14 | true | 2000-05-19 | 2016-10-10 09:30:20.0 | Bangalore | MAINS |
> | 18 | | 3 | 7.86786786787E9 | true | 1980-10-05 | 1995-10-07 19:30:20.0 | Bangalore | IT |
> | 19 | | 2 | 5464545.33 | true | 1986-06-06 | 2008-08-14 22:30:20.0 | Delhi | DATA |
> | 20 | NULL | 3 | 7867867.34 | true | 2000-05-01 | 2014-01-18 09:30:20.0 | Bangalore | MAINS |
> +-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
> 13 rows selected (1.775 seconds)
> *0: /> select * from external_primitive limit 14;* --failed getting number format exception;
> INFO : Execution ID: 97
> *java.lang.NumberFormatException: For input string: "776"*
>  at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>  at java.lang.Integer.parseInt(Integer.java:569)
>  at java.lang.Integer.parseInt(Integer.java:615)
>  at java.sql.Date.valueOf(Date.java:133)
>  at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
>  at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
>  at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
>  at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
>  at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
>  at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
>  at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
>  at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
>  at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
>  at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
>  at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
>  at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
>  at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
> *Error: Unrecognized column type:DATE_TYPE (state=,code=0)*
> *0: /> select min(dob) from external_primitive;* --failed getting number format exception;
> INFO : Execution ID: 99
> *java.lang.NumberFormatException: For input string: "776"*
>  at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>  at java.lang.Integer.parseInt(Integer.java:569)
>  at java.lang.Integer.parseInt(Integer.java:615)
>  at java.sql.Date.valueOf(Date.java:133)
>  at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
>  at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
>  at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
>  at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
>  at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
>  at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
>  at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
>  at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
>  at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
>  at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
>  at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
>  at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
>  at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
> *Error: Unrecognized column type:DATE_TYPE (state=,code=0)*
> *0: /> select max(dob) from external_primitive;*
> INFO : Execution ID: 101
> +-------------+
> | max(dob) |
> +-------------+
> | 2000-05-19 |
> +-------------+
> 1 row selected (1.875 seconds)
> *0: /> select dob from external_primitive;* --failed getting number format exception;
> INFO : Execution ID: 103
> *java.lang.NumberFormatException: For input string: "776"*
>  at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>  at java.lang.Integer.parseInt(Integer.java:569)
>  at java.lang.Integer.parseInt(Integer.java:615)
>  at java.sql.Date.valueOf(Date.java:133)
>  at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
>  at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
>  at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
>  at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
>  at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
>  at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
>  at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
>  at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
>  at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
>  at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
>  at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
>  at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
>  at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
>  at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
> *Error: Unrecognized column type:DATE_TYPE (state=,code=0)*
>  
>  ** 

--
This message was sent by Atlassian Jira
(v8.3.4#803005)