Re: Support updating/deleting data for stream table

Posted by sraghunandan on May 31, 2018; 4:16am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Support-updating-deleting-data-for-stream-table-tp51060p51173.html

Hi,
But this leads to inconsistent data returned to the user. For example user
wanted to replace all 2g with 4g,but in stream segments it will still be 2g
and would be returned in user query. I think we need to handle this
scenario also

Regards
Raghu

On Wed, 30 May 2018, 9:22 pm Liang Chen, <[hidden email]> wrote:

> Hi
>
> Thank you started this discussion thread.
> Agree with solution1, use the easy way to delete data for stream table.
>
> Regards
> Liang
>
> xm_zzc wrote
> > Hi dev:
> >   Sometimes we need to delete some historical data from stream table to
> > make
> > the table size not too large, but currently the stream table can't
> support
> > updating/deleting data, so we need to stop the app and use 'alter table
> > COMPACT 'close_streaming' command to close stream table, and then delete
> > data.
> >   According to discussion with Jacky and David offline, there are two
> > solutions to resolve this without stopping app:
> >
> >   1. set all
> *
> > non-stream
> *
> >  segments to 'carbon.input.segments.tablename'
> > property to delete data except stream segment, this's easy to implement,
> > but not very precise when there are data stored in stream segments.
> >   2. support deleting data for stream segment too, this's more
> > complicated, but precise.
> >
> >   I think we can implement with solution 1 first, and then consider the
> > implementation of solution 2 in depth.
> >
> >   Welcome to feedback, thanks.
> >
> >
> >
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>