[jira] [Resolved] (CARBONDATA-697) single_pass is not used while doing data load

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ravindra Pesala resolved CARBONDATA-697.
----------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.1.0-incubating

> single_pass is not used while doing data load
> ---------------------------------------------
>
>                 Key: CARBONDATA-697
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-697
>             Project: CarbonData
>          Issue Type: Bug
>    Affects Versions: 0.1.1-incubating
>         Environment: Spark 2.1
>            Reporter: Ramakrishna
>             Fix For: 1.1.0-incubating
>
>          Time Spent: 1h 40m
>  Remaining Estimate: 0h
>
> 1. 1st load with 'USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE'
> 2. 2nd load with 'USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE'
> Expected: GlobalDictionaryUtil should not be triggered in the 2nd time load
> Actual :  GlobalDictionaryUtil was triggered
> Job Id (Job Group)  ▾
> Description
> Submitted
> Duration
> Stages: Succeeded/Total Tasks (for all stages): Succeeded/Total
> 3 (891a750d-1e08-432f-bcf4-5501332d6d09)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table flow_carbon options('USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE','DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at CarbonDataRDDFactory.scala:651 2017/02/06 15:23:05 41 s 1/1
> 3/3
> 2 (891a750d-1e08-432f-bcf4-5501332d6d09)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table flow_carbon options('USE_KETTLE'='FALSE','SINGLE_PASS'='TRUE','DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at GlobalDictionaryUtil.scala:742 2017/02/06 15:22:29 36 s 2/2
> 42/42
> 1 (c7706941-8184-40ab-9ceb-712af96e15e3)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table flow_carbon options('USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE','DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at CarbonDataRDDFactory.scala:651 2017/02/06 15:21:42 47 s 1/1
> 3/3
> 0 (c7706941-8184-40ab-9ceb-712af96e15e3)
> LOAD DATA inpath 'hdfs://hacluster/commbank/splits/FG2' into table flow_carbon options('USE_KETTLE'='FALSE','SINGLE_PASS'='FALSE','DELIMITER'=',', 'QUOTECHAR'='"','FILEHEADER'='aco_ac,ac_dte,txn_cnt,jrn_par,mfm_jrn_no,cbn_jrn_no,ibs_jrn_no,vch_no,vch_seq,srv_cde,cus_no,bus_cd_no,id_flg,cus_ac,bv_cde,bv_no,txn_dte,txn_time,txn_tlr,txn_bk,txn_br,ety_tlr,ety_bk,ety_br,bus_pss_no,chk_flg,chk_tlr,chk_jrn_no,bus_sys_no,bus_opr_cde,txn_sub_cde,fin_bus_cde,fin_bus_sub_cde,opt_prd_cde,chl,tml_id,sus_no,sus_seq,cho_seq,itm_itm,itm_sub,itm_sss,dc_flg,amt,bal,ccy,spv_flg,vch_vld_dte,pst_bk,pst_br,ec_flg,aco_tlr,opp_ac,opp_ac_nme,opp_bk,gen_flg,his_rec_sum_flg,his_flg,vch_typ,val_dte,opp_ac_flg,cmb_flg,ass_vch_flg,cus_pps_flg,bus_rmk_cde,vch_bus_rmk,tec_rmk_cde,vch_tec_rmk,rsv_ara,own_br,own_bk,gems_last_upd_d,gems_last_upd_d_bat,maps_date,maps_job,dt')
> collect at GlobalDictionaryUtil.scala:742

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)