Posted by
ravipesala on
Dec 03, 2019; 10:06am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Propose-feature-change-in-CarbonData-2-0-tp87540p87707.html
Hi,
Thank you for proposing. Please check my comments below.
1.Global dictionary: It was one of the prime features when it was initially
released to apache. Even though spark has introduced tungsten still it has
its benefits like compression, filtering and aggregation queries. But after
the introduction of a local dictionary, it got solved partially like
compression and filtering (cannot get the same performance as a global
dictionary). But only the major drawback here is the data load performance.
In some cases like MOLAP cube (build once) it is still might be useful.
Vote: 0
2. Bucket: It is a very useful feature if we use it. if we are planning to
remove better find the alternative to this feature first. Since these
features are available in spark+parquet it would be helpful for users who
want to migrate to carbon. As I know this feature was never productized and
it is still in experimental. So if we are planning to keep it better make it
productize. Vote : -1
3. Carbon custom partition: Vote : +1
4. Batch Sort : Vote : +1
5. Page level inverse index : It makes the store size bigger to store these
indexes. It is really helpful in case of multiple in filters but it is got
overshadowed by its IO and CPU performance due to its size. Vote : +1
5. old preaggregate and time series datamap implementation : Vote : +1
(remove pre-aggregate)
6. Lucene DataMap: It is a helpful feature but I guess it had performance
issues due to bad integration. It would be better if we can fix these issues
instead of removing it. Moreover, it is a separate module so there would not
be any code maintenance problem. Vote : -1
7. STORED BY : Vote : +1
refractory points:
1 & 2 : I think at this point of time it would be a massive refractory but
very less outcome. So better don't do it. Vote : -1
3 &4 : Vote : +1
Regards,
Ravindra.
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/