Unable to perform compaction,

[prabhatkashyap]

Hello,
There is some issue with compaction. Auto and force compaction are not working.
In spark logs I got this error:

Copying /tmp/5609139520219/0/database/bva/Fact/Part0/Segment_0.1/0/part-0-0-1476956184000.carbondata --> hdfs://localhost:54310/user/hive/warehouse/carbon.store/database/bva/Fact/Part0/Segment_0.1
ERROR 20-10 15:06:26,009 - [Executor task launch worker-0][partitionID:bva;queryID:5609201850014] Exception while closing the handler in compaction merger Problem while copying file from local store to carbon store

[Liang Chen]

Hi

Can you provide the detail test steps and error logs

Regards
Liang


2016-10-20 19:35 GMT+08:00 prabhatkashyap <[hidden email]>:

> Hello,
> There is some issue with compaction. Auto and force compaction are not
> working.
> In spark logs I got this error:
>
>
>
>
>
>
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/Unable-to-
> perform-compaction-tp2099.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>

--

Regards
Liang

[prabhatkashyap]

Hi Liang,

Sorry for the late reply,

For auto compaction:

I've set my default threshold for compaction and set

##To enable compaction while data loading
carbon.enable.auto.load.merge=true

and for force compaction I've used the ALTER Query

ALTER TABLE <table_name> COMPACT 'minor'
ALTER TABLE <table_name> COMPACT 'major'

But on both of the cases it is showing me error

INFO  26-10 11:53:12,763 - [Executor task launch worker-16][partitionID:one;queryID:7350148251967] Copying /tmp/7350124040737/0/lastday/one/Fact/Part0/Segment_0.1/0/part-0-0-1477462991000.carbondata --> hdfs://localhost:54310/Opt/CarbonStore/lastday/one/Fact/Part0/Segment_0.1
INFO  26-10 11:53:12,763 - [Executor task launch worker-16][partitionID:one;queryID:7350148251967] The configured block size is 0 Byte, the actual carbon file size is 1 KB, choose the max value 1 KB as the block size on HDFS
ERROR 26-10 11:53:12,776 - [Executor task launch worker-16][partitionID:one;queryID:7350148251967] Exception while closing the handler in compaction merger Problem while copying file from local store to carbon store
INFO  26-10 11:53:12,777 - [Executor task launch worker-16][partitionID:one;queryID:7350148251967] Deleted the local store location/tmp/7350124040737/0