Support updating/deleting data for stream table

Posted by xm_zzc on May 29, 2018; 5:10pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Support-updating-deleting-data-for-stream-table-tp51060.html

Hi dev:
  Sometimes we need to delete some historical data from stream table to make
the table size not too large, but currently the stream table can't support
updating/deleting data, so we need to stop the app and use 'alter table
COMPACT 'close_streaming' command to close stream table, and then delete
data.
  According to discussion with Jacky and David offline, there are two
solutions to resolve this without stopping app:
 
  1. set all non-stream segments to 'carbon.input.segments.tablename'
property to delete data except stream segment, this's easy to implement, but not very precise when there are data stored in stream segments.
  2. support deleting data for stream segment too, this's more complicated, but precise.
 
  I think we can implement with solution 1 first, and then consider the
implementation of solution 2 in depth.
 
  Welcome to feedback, thanks.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/