Posted by
geda on
Jan 04, 2017; 8:51am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/how-to-make-carbon-run-faster-tp5305p5451.html
thanks,alter compact not work,i change to set auto.load
error is the same
INFO 04-01 16:47:20,676 - main compaction need status is true
AUDIT 04-01 16:47:20,676 - [datanode07.bi][qixing][Thread-1]Compaction
request received for table test
INFO 04-01 16:47:20,679 - main Acquired the compaction lock.
INFO 04-01 16:47:20,693 - main loads identified for merge is 0
INFO 04-01 16:47:20,693 - main loads identified for merge is 1
INFO 04-01 16:47:20,693 - main loads identified for merge is 2
INFO 04-01 16:47:20,693 - main loads identified for merge is 3
INFO 04-01 16:47:20,693 - main loads identified for merge is 4
INFO 04-01 16:47:20,693 - main loads identified for merge is 5
INFO 04-01 16:47:20,706 - pool-24-thread-1 spark.executor.instances
property is set to =8
INFO 04-01 16:47:20,806 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,812 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,821 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,827 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,834 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,840 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,846 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
INFO 04-01 16:47:20,852 - pool-24-thread-1 ************************Total
Number Rows In BTREE: 1
ERROR 04-01 16:47:20,859 - main Exception in compaction thread
java.io.IOException: java.lang.NullPointerException
2017-01-04 12:28 GMT+08:00 QiangCai <
[hidden email]>:
> HI,
> Total memory of cluster is 16G*40 = 640GB, total size of data file is
> 600/1024GB*60 = 35GB, so I suggest to load all data files(60 days) once.
>
> BTW, if loading small data file(total size) for each data loading, it will
> generate small carbon files.
> You can add "carbon.enable.auto.load.merge=true" to carbon.properties file
> to enable compaction feature.
>
>
>
> --
> View this message in context:
http://apache-carbondata-> mailing-list-archive.1130556.n5.nabble.com/how-to-make-
> carbon-run-faster-tp5305p5427.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>