Posted by
xubo245 on
Oct 31, 2018; 1:49am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-Refactor-dynamic-configuration-tp67057.html
*Background:*
In CarbonData, there are many configuration in:
org.apache.carbondata.core.constants.CarbonCommonConstants,
org.apache.carbondata.core.constants.CarbonV3DataFormatConstants,
org.apache.carbondata.core.constants.CarbonLoadOptionConstants;
and so on. Which one can be dynamic configure, which one can not be dynamic
configure, it's not clear.
User will confuse when they use the CarbonData configuration. So we should
optimize it.
*Plan:*
Refactor dynamic configuration for carbon:
1. Decide and collect all dynamic configurations which can be SET in
carbondata application like in beeline.
2. For every dynamic configuration, use an annotation to tag them. (re-use
the CarbonProperty annotation but change its name). This annotation should
be used for validation when user invoking SET command.
*Implement:*
1.change CarbonProperty annotation to DynamicConfigurable, add
NonDynamicConfigurable, DefaultValue,Threshold annotation and so on
2.unify default value variable name: XX_XX_DEFAULT
CARBON_MERGE_SORT_READER_THREAD_DEFAULTVALUE
=>CARBON_MERGE_SORT_READER_THREAD_DEFAULT
SORT_SIZE_DEFAULT_VAL
=>SORT_SIZE_DEFAULT
CARBON_TIMESTAMP_DEFAULT_FORMAT =>
CARBON_TIMESTAMP_FORMAT_DEFAULT
DEFAULT_COMPRESSOR =>
COMPRESSOR_DEFAULT
3.optimize the properties: remove some warning, such as:
carbon.query.validate.directqueryondatamap =>
carbon.query.validate.direct.query.on.datamap
carbon.scheduler.minregisteredresourcesratio=>
carbon.scheduler.min.registered.resources.ratio
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/