Re:Propose feature change in CarbonData 2.0

Posted by xm_zzc on Dec 03, 2019; 4:19pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Propose-feature-change-in-CarbonData-2-0-tp87540p87720.html

Hi:
  Thank you for proposing. My votes are below:

  1,3,4,5.1,5.2,7:  +1
  2:                      0
  6:                      -1, but should be optimzied.

  And there are some internal refactory we can do:
  1. Unify dimension and measure   +1.

  2. Keep the column order the same as schema order   0.

  3. Spark integration refactory based on Spark extension interface   +1

  4. Store optimization PR2729   +1

  In my opinion, we also can do some  refactor:
  1. there are many places using string[] to store data in the process of loading data, it can replace with InternalRow object to save  memory;
  2. remove 'streaming' property and eliminate the difference between streaming and batch table, users can insert data into a table by batch way and streaming way.