Login  Register

Re: [Design Discussion] Transaction manager, time travel and segment interface refactoring

Posted by Venkata Gollamudi on May 02, 2021; 4:37pm
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Design-Discussion-Transaction-manager-time-travel-and-segment-interface-refactoring-tp107770p108010.html

+1

Valuable feature for data consistency across tables, mvs, indexes.

On Wed, Apr 28, 2021 at 7:02 PM Ravindra Pesala <[hidden email]>
wrote:

> +1
>
> Much needed feature and interface refactoring. Thanks for working on it.
>
> Regards,
> Ravindra.
>
> On Thu, 22 Apr 2021 at 2:36 PM, Ajantha Bhat <[hidden email]>
> wrote:
>
> > Hi All,
> > In this thread, I am continuing the below discussion along with the
> > Transaction Manager and Time Travel feature design.
> >
> >
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/Discussion-Taking-the-inputs-for-Segment-Interface-Refactoring-td101950.html
> >
> > The goal of this requirement is as follows.
> >
> >    1.
> >
> >    Implement a “Transaction Manager” with optimistic concurrency to
> provide
> >    within a table transaction/versioning. (interfaces should also be
> >    flexible enough to support across table transactions)
> >    2.
> >
> >    Support time travel in carbonData.
> >    3.
> >
> >    Decouple and clean up segment interfaces. (which should also help in
> >    supporting segment concepts to other open formats under carbonData
> > metadata
> >    service)
> >
> >
> > The design document is attached in JIRA.
> > JIRA link: https://issues.apache.org/jira/browse/CARBONDATA-4171
> > GoogleDrive link:
> >
> >
> https://docs.google.com/document/d/1FsVsXjj5QCuFDrzrayN4Qo0LqWc0Kcijc_jL7pCzfXo/edit?usp=sharing
> >
> > Please have a look. suggestions are welcome.
> > I have mentioned some TODO in the document, I will be updating it in the
> V2
> > version soon.
> > Implementation will be done by adding subtasks under the same JIRA.
> >
> > Thanks,
> > Ajantha
> >
> --
> Thanks & Regards,
> Ravi
>