[jira] [Commented] (CARBONDATA-3908) When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values.

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-3908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17303162#comment-17303162 ]

Ajantha Bhat commented on CARBONDATA-3908:
------------------------------------------

[https://github.com/apache/carbondata/pull/4001]

 

> When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values.
> -------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-3908
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3908
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 2.0.0
>         Environment: FI cluster and opensource cluster.
>            Reporter: Prasanna Ravichandran
>            Priority: Major
>             Fix For: 2.1.1
>
>
> When a carbon segment is added through the alter add segments query, then it is not accounting the added carbon segment values. If we do count(*) on the added segment, then it is always showing as 0.
> Test queries:
> drop table if exists uniqdata;
> CREATE TABLE uniqdata (cust_id int,cust_name String,active_emui_version string, dob timestamp, doj timestamp, bigint_column1 bigint,bigint_column2 bigint,decimal_column1 decimal(30,10), decimal_column2 decimal(36,36),double_column1 double, double_column2 double,integer_column1 int) stored as carbondata;
> load data inpath 'hdfs://hacluster/BabuStore/Data/2000_UniqData.csv' into table uniqdata options('fileheader'='cust_id,cust_name,active_emui_version,dob,doj,bigint_column1,bigint_column2,decimal_column1,decimal_column2,double_column1,double_column2,integer_column1','bad_records_action'='force');
> --hdfs dfs -mkdir /uniqdata-carbon-segment;
> --hdfs dfs -cp /user/hive/warehouse/uniqdata/Fact/Part0/Segment_0/* /uniqdata-carbon-segment/
> Alter table uniqdata add segment options ('path'='hdfs://hacluster/uniqdata-carbon-segment/','format'='carbon');
> select count(*) from uniqdata;--4000 expected as one load of 2000 records happened and same segment is added again;
> set carbon.input.segments.default.uniqdata=1;
> select count(*) from uniqdata;--2000 expected - it should just show the records count of added segments;
> CONSOLE:
> /> set carbon.input.segments.default.uniqdata=1;
> +-----------------------------------------+--------+
> | key | value |
> +-----------------------------------------+--------+
> | carbon.input.segments.default.uniqdata | 1 |
> +-----------------------------------------+--------+
> 1 row selected (0.192 seconds)
> /> select count(*) from uniqdata;
> INFO : Execution ID: 1734
> +-----------+
> | count(1) |
> +-----------+
> | 2000 |
> +-----------+
> 1 row selected (4.036 seconds)
> /> set carbon.input.segments.default.uniqdata=2;
> +-----------------------------------------+--------+
> | key | value |
> +-----------------------------------------+--------+
> | carbon.input.segments.default.uniqdata | 2 |
> +-----------------------------------------+--------+
> 1 row selected (0.088 seconds)
> /> select count(*) from uniqdata;
> INFO : Execution ID: 1745
> +-----------+
> | count(1) |
> +-----------+
> | 2000 |
> +-----------+
> 1 row selected (6.056 seconds)
> /> set carbon.input.segments.default.uniqdata=3;
> +-----------------------------------------+--------+
> | key | value |
> +-----------------------------------------+--------+
> | carbon.input.segments.default.uniqdata | 3 |
> +-----------------------------------------+--------+
> 1 row selected (0.161 seconds)
> /> select count(*) from uniqdata;
> INFO : Execution ID: 1753
> +-----------+
> | count(1) |
> +-----------+
> | 0 |
> +-----------+
> 1 row selected (4.875 seconds)
> /> show segments for table uniqdata;
> +-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+
> | ID | Status | Load Start Time | Load Time Taken | Partition | Data Size | Index Size | File Format |
> +-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+
> | 4 | Success | 2020-07-17 16:01:53.673 | 5.579S | {} | 269.10KB | 7.21KB | columnar_v3 |
> | 3 | Success | 2020-07-17 16:00:24.866 | 0.578S | {} | 88.55KB | 1.81KB | columnar_v3 |
> | 2 | Success | 2020-07-17 15:07:54.273 | 0.642S | {} | 36.72KB | NA | orc |
> | 1 | Success | 2020-07-17 15:03:59.767 | 0.564S | {} | 89.26KB | NA | parquet |
> | 0 | Success | 2020-07-16 12:44:32.095 | 4.484S | {} | 88.55KB | 1.81KB | columnar_v3 |
> +-----+----------+--------------------------+------------------+------------+------------+-------------+--------------+
> Expected result: Records added by adding carbon segment should be considered.
> Actual result: Records added by adding carbon segment is not considered.

--
This message was sent by Atlassian Jira
(v8.3.4#803005)