Apache CarbonData Dev Mailing List archive › Apache CarbonData JIRA issues

[jira] [Updated] (CARBONDATA-4029) After delete in the table which has Alter-added SDK segments, then the count(*) is 0.

classic Classic list List threaded Threaded
♦
♦
 1 message Options
Akash R Nilugal (Jira)
Reply | Threaded
Open this post in threaded view
♦
♦
|

[jira] [Updated] (CARBONDATA-4029) After delete in the table which has Alter-added SDK segments, then the count(*) is 0.

Akash R Nilugal (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-4029?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Prasanna Ravichandran updated CARBONDATA-4029:
----------------------------------------------
    Description:     (was: We are getting Number format exception while querying on the date columns. Attached the SDK files also.

Test queries:

--SDK compaction;
 drop table if exists external_primitive;
 create table external_primitive (id int, name string, rank smallint, salary double, active boolean, dob date, doj timestamp, city string, dept string) stored as carbondata;
 alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive','format'='carbon');
 alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive2','format'='carbon');
 alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive3','format'='carbon');
 alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive4','format'='carbon');
 alter table external_primitive add segment options('path'='hdfs://hacluster/sdkfiles/primitive5','format'='carbon');
 
 alter table external_primitive compact 'minor'; --working fine pass;
 select count(*) from external_primitive;--working fine pass;

show segments for table external_primitive;
 select * from external_primitive limit 13; --working fine pass;
 select * from external_primitive limit 14; --failed getting number format exception;
select min(dob) from external_primitive; --failed getting number format exception;
select max(dob) from external_primitive; --working;
select dob from external_primitive; --failed getting number format exception;

Console:

*0: /> show segments for table external_primitive;*
+------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
| ID | Status | Load Start Time | Load Time Taken | Partition | Data Size | Index Size | File Format |
+------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
| 4 | Success | 2020-10-13 11:52:04.012 | 0.511S | {} | 1.88KB | 655.0B | columnar_v3 |
| 3 | Compacted | 2020-10-13 11:52:00.587 | 0.828S | {} | 1.88KB | 655.0B | columnar_v3 |
| 2 | Compacted | 2020-10-13 11:51:57.767 | 0.775S | {} | 1.88KB | 655.0B | columnar_v3 |
| 1 | Compacted | 2020-10-13 11:51:54.678 | 1.024S | {} | 1.88KB | 655.0B | columnar_v3 |
| 0.1 | Success | 2020-10-13 11:52:05.986 | 5.785S | {} | 9.62KB | 5.01KB | columnar_v3 |
| 0 | Compacted | 2020-10-13 11:51:51.072 | 1.125S | {} | 8.55KB | 4.25KB | columnar_v3 |
+------+------------+--------------------------+------------------+------------+------------+-------------+--------------+
6 rows selected (0.45 seconds)
*0: /> select * from external_primitive limit 13;* --working fine pass;
INFO : Execution ID: 95
+-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
| id | name | rank | salary | active | dob | doj | city | dept |
+-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
| 1 | AAA | 3 | 3444345.66 | true | 1979-12-09 | 2011-02-09 22:30:20.0 | Pune | IT |
| 2 | BBB | 2 | 543124.66 | false | 1987-02-19 | 2017-01-01 09:30:20.0 | Bangalore | DATA |
| 3 | CCC | 1 | 787878.888 | false | 1982-05-12 | 2015-11-30 23:50:20.0 | Pune | DATA |
| 4 | DDD | 1 | 99999.24 | true | 1981-04-09 | 2000-01-15 04:30:20.0 | Delhi | MAINS |
| 5 | EEE | 3 | 545656.99 | true | 1987-12-09 | 2017-11-25 01:30:20.0 | Delhi | IT |
| 6 | FFF | 2 | 768678.0 | false | 1987-12-20 | 2017-01-10 02:30:20.0 | Bangalore | DATA |
| 7 | GGG | 3 | 765665.0 | true | 1983-06-12 | 2016-12-31 23:30:20.0 | Pune | IT |
| 8 | HHH | 2 | 567567.66 | false | 1979-01-12 | 1995-01-01 09:30:20.0 | Bangalore | DATA |
| 9 | III | 2 | 787878.767 | true | 1985-02-19 | 2005-08-14 22:30:20.0 | Pune | DATA |
| 10 | JJJ | 3 | 887877.14 | true | 2000-05-19 | 2016-10-10 09:30:20.0 | Bangalore | MAINS |
| 18 | | 3 | 7.86786786787E9 | true | 1980-10-05 | 1995-10-07 19:30:20.0 | Bangalore | IT |
| 19 | | 2 | 5464545.33 | true | 1986-06-06 | 2008-08-14 22:30:20.0 | Delhi | DATA |
| 20 | NULL | 3 | 7867867.34 | true | 2000-05-01 | 2014-01-18 09:30:20.0 | Bangalore | MAINS |
+-----+-------+-------+------------------+---------+-------------+------------------------+------------+--------+
13 rows selected (1.775 seconds)
*0: /> select * from external_primitive limit 14;* --failed getting number format exception;
INFO : Execution ID: 97
*java.lang.NumberFormatException: For input string: "776"*
 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:569)
 at java.lang.Integer.parseInt(Integer.java:615)
 at java.sql.Date.valueOf(Date.java:133)
 at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
 at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
 at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
 at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
 at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
 at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
 at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
 at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
 at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
 at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
 at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
 at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
 at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
 at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
 at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
*Error: Unrecognized column type:DATE_TYPE (state=,code=0)*
*0: /> select min(dob) from external_primitive;* --failed getting number format exception;
INFO : Execution ID: 99
*java.lang.NumberFormatException: For input string: "776"*
 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:569)
 at java.lang.Integer.parseInt(Integer.java:615)
 at java.sql.Date.valueOf(Date.java:133)
 at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
 at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
 at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
 at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
 at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
 at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
 at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
 at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
 at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
 at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
 at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
 at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
 at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
 at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
 at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
*Error: Unrecognized column type:DATE_TYPE (state=,code=0)*
*0: /> select max(dob) from external_primitive;*
INFO : Execution ID: 101
+-------------+
| max(dob) |
+-------------+
| 2000-05-19 |
+-------------+
1 row selected (1.875 seconds)
*0: /> select dob from external_primitive;* --failed getting number format exception;
INFO : Execution ID: 103
*java.lang.NumberFormatException: For input string: "776"*
 at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
 at java.lang.Integer.parseInt(Integer.java:569)
 at java.lang.Integer.parseInt(Integer.java:615)
 at java.sql.Date.valueOf(Date.java:133)
 at org.apache.hive.jdbc.HiveBaseResultSet.evaluate(HiveBaseResultSet.java:448)
 at org.apache.hive.jdbc.HiveBaseResultSet.getColumnValue(HiveBaseResultSet.java:424)
 at org.apache.hive.jdbc.HiveBaseResultSet.getString(HiveBaseResultSet.java:537)
 at org.apache.hive.beeline.Rows$Row.<init>(Rows.java:166)
 at org.apache.hive.beeline.BufferedRows.<init>(BufferedRows.java:54)
 at org.apache.hive.beeline.IncrementalRowsWithNormalization.<init>(IncrementalRowsWithNormalization.java:50)
 at org.apache.hive.beeline.BeeLine.print(BeeLine.java:2195)
 at org.apache.hive.beeline.Commands.executeInternal(Commands.java:1009)
 at org.apache.hive.beeline.Commands.execute(Commands.java:1208)
 at org.apache.hive.beeline.Commands.sql(Commands.java:1137)
 at org.apache.hive.beeline.BeeLine.dispatch(BeeLine.java:1315)
 at org.apache.hive.beeline.BeeLine.execute(BeeLine.java:1179)
 at org.apache.hive.beeline.BeeLine.begin(BeeLine.java:1034)
 at org.apache.hive.beeline.BeeLine.mainWithInputRedirection(BeeLine.java:519)
 at org.apache.hive.beeline.BeeLine.main(BeeLine.java:501)
*Error: Unrecognized column type:DATE_TYPE (state=,code=0)*

 

 ** )

> After delete in the table which has Alter-added SDK segments, then the count(*) is 0.
> -------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-4029
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-4029
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>         Environment: 3 node FI cluster
>            Reporter: Prasanna Ravichandran
>            Priority: Minor
>         Attachments: Primitive.rar
>
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Free forum by Nabble Edit this page