[
https://issues.apache.org/jira/browse/CARBONDATA-2021?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Akash R Nilugal updated CARBONDATA-2021:
----------------------------------------
Description:
when delete is success and update is failed while writing status file then a stale carbon data file is created.
so removing that file on clean up . and also not considering that one during query.
when the update operation is running and the user stops it abruptly,
then the carbon data file will be remained in the store .
so extra data is coming.
during the next update the clean up of the files need to be handled.
and in query also new data file should be excluded.
was:
when delete is success and update is failed while writing status file then a stale carbon data file is created.
so removing that file on clean up . and also not considering that one during query.
when the update operation is running and the user stops it abruptly,
then the carbon data file will be remained in the store .
so extra data is coming.
during the next update the clean up of the files need to be handled.
and in query also new data file should be excluded.
CREATE TABLE uniqdata_carbon1 (CUST_ID int,CUST_NAME String,ACTIVE_EMUI_VERSION string, DOB timestamp, DOJ timestamp, BIGINT_COLUMN1 bigint,BIGINT_COLUMN2 bigint,DECIMAL_COLUMN1 decimal(30,10), DECIMAL_COLUMN2 decimal(36,10),Double_COLUMN1 double, Double_COLUMN2 double, INTEGER_COLUMN1 int)stored by 'carbondata';
LOAD DATA INPATH 'hdfs://hacluster/chetan/split3.csv' into table uniqdata_carbon1 OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','BAD_RECORDS_ACTION'='FORCE','FILEHEADER'='CUST_ID,CUST_NAME,ACTIVE_EMUI_VERSION,DOB,DOJ,BIGINT_COLUMN1,BIGINT_COLUMN2,DECIMAL_COLUMN1,DECIMAL_COLUMN2,Double_COLUMN1,Double_COLUMN2,INTEGER_COLUMN1');
create table uniqdata_carbon stored by 'carbondata' location '/opt/external_location' as select * from uniqdata_carbon1;
create table uniqdata_carbon stored by 'carbondata' tblproperties('sort_columns'='CUST_ID') as select * from uniqdata_carbon1;
199221,CUST_NAME_190221,ACTIVE_EMUI_VERSION_190221,2010-10-04 02:57:17,2012-10-04 03:56:07,123372000000,-2.2337200E+11,12345705900,22345705900,11234567490,-11234567490,27000
> when delete is success and update is failed while writing status file then a stale carbon data file is created.
> ----------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-2021
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-2021> Project: CarbonData
> Issue Type: Bug
> Reporter: Akash R Nilugal
> Assignee: Akash R Nilugal
> Priority: Minor
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> when delete is success and update is failed while writing status file then a stale carbon data file is created.
> so removing that file on clean up . and also not considering that one during query.
> when the update operation is running and the user stops it abruptly,
> then the carbon data file will be remained in the store .
> so extra data is coming.
> during the next update the clean up of the files need to be handled.
> and in query also new data file should be excluded.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
Locked
