[jira] [Created] (CARBONDATA-3908) When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values.

[Akash R Nilugal (Jira)]

Prasanna Ravichandran created CARBONDATA-3908:
-------------------------------------------------

             Summary: When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values.
                 Key: CARBONDATA-3908
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3908
             Project: CarbonData
          Issue Type: Bug
    Affects Versions: 2.0.0
         Environment: FI cluster and opensource cluster.
            Reporter: Prasanna Ravichandran


When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values. If we do count(*) on the added segment, then it is always showing as 0.

Test queries:

drop table if exists uniqdata;
CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2 bigint,decimal_column1 decimal(30,10), decimal_column2 decimal(36,36),double_column1 double, double_column2 double,integer_column1 int) stored as carbondata;
load data inpath 'hdfs://hacluster/BabuStore/Data/2000_UniqData.csv' into table uniqdata options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');

--hdfs dfs -mkdir /uniqdata-carbon-segment;
--hdfs dfs -cp /user/hive/warehouse/uniqdata/Fact/Part0/Segment_0/* /uniqdata-carbon-segment/
Alter table uniqdata add segment options ('path'='hdfs://hacluster/uniqdata-carbon-segment/','format'='carbon');

select count(*) from uniqdata;--4000 expected as one load of 2000 records happened and same segment is added again;

set carbon.input.segments.default.uniqdata=1;
select count(*) from uniqdata;--2000 expected - it should just show the records count of added segments;

CONSOLE:

/> set carbon.input.segments.default.uniqdata=1;
+-----------------------------------------+--------+
| key | value |
+-----------------------------------------+--------+
| carbon.input.segments.default.uniqdata | 1 |
+-----------------------------------------+--------+
1 row selected (0.192 seconds)
/> select count(*) from uniqdata;
INFO : Execution ID: 1734
+-----------+
| count(1) |
+-----------+
| 2000 |
+-----------+
1 row selected (4.036 seconds)
/> set carbon.input.segments.default.uniqdata=2;
+-----------------------------------------+--------+
| key | value |
+-----------------------------------------+--------+
| carbon.input.segments.default.uniqdata | 2 |
+-----------------------------------------+--------+
1 row selected (0.088 seconds)
/> select count(*) from uniqdata;
INFO : Execution ID: 1745
+-----------+
| count(1) |
+-----------+
| 2000 |
+-----------+
1 row selected (6.056 seconds)
/> set carbon.input.segments.default.uniqdata=3;
+-----------------------------------------+--------+
| key | value |
+-----------------------------------------+--------+
| carbon.input.segments.default.uniqdata | 3 |
+-----------------------------------------+--------+
1 row selected (0.161 seconds)
/> select count(*) from uniqdata;
INFO : Execution ID: 1753
+-----------+
| count(1) |
+-----------+
| 0 |
+-----------+
1 row selected (4.875 seconds)
/> show segments for table uniqdata;
+-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+
| ID | Status | Load Start Time | Load Time Taken | Partition | Data Size | Index Size | File Format |
+-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+
| 4 | Success | 2020-07-17 16:01:53.673 | 5.579S | {} | 269.10KB | 7.21KB | columnar_v3 |
| 3 | Success | 2020-07-17 16:00:24.866 | 0.578S | {} | 88.55KB | 1.81KB | columnar_v3 |
| 2 | Success | 2020-07-17 15:07:54.273 | 0.642S | {} | 36.72KB | NA | orc |
| 1 | Success | 2020-07-17 15:03:59.767 | 0.564S | {} | 89.26KB | NA | parquet |
| 0 | Success | 2020-07-16 12:44:32.095 | 4.484S | {} | 88.55KB | 1.81KB | columnar_v3 |
+-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+

Expected result: Records added by adding carbon segment should be considered.

Actual result: Records added by adding carbon segment is not considered.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)