[Discussion] Parallel Insert and Update

[Kejian Li]

Dear community,

This mail is regarding parallel insert and update. Currently we are not
supporting concurrent insert (or load data) and update because it may lead to
data inconsistency or incorrect result.

Now Carbon blocks update operation when insert operation is in progress by
throwing out concurrent operation exception directly. If there is an
executing insert operation that is very time consuming, then update
operation has to wait and sometimes this waiting time is very long.

To come out with this problem, we are planning to support parallel insert
and update. And here I have proposed one of the solutions to implement this
feature.

This is the design document:  Parallel_Insert_and_Update.pdf

Please go through this solution document and provide your input if this
approach is okay or any drawback is there.

Thanks & Regards
Kejian Li

[Ajantha Bhat]

Hi Kejian Li,
Thanks for working on this.

I see that this design and requirement is similar to what Nihal has
discussed a few days ago.
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/DISCUSSION-Parallel-compaction-and-update-td100338.html

So, Probably as Ravidra suggested for Nihal, maybe better to handle this by
segment interface refactoring for this also.

Thanks,
Ajantha

On Wed, Oct 14, 2020 at 2:46 PM Kejian Li <[hidden email]> wrote:

> Dear community,
>
> This mail is regarding parallel insert and update. Currently we are not
> supporting concurrent insert (or load data) and update because it may cause
> data inconsistency or incorrect result.
>
> Now Carbon blocks update operation when insert operation is in progress by
> throwing out concurrent operation exception directly. If there is an
> executing insert operation that is very time consuming, then update
> operation has to wait and sometimes this waiting time is very long.
>
> To come out with this problem, we are planning to support parallel insert
> and update. And here I have proposed one of the solutions to implement this
> feature.
>
> This is the design document:  Parallel_Insert_and_Update.pdf
> <
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t495/Parallel_Insert_and_Update.pdf>
>
>
> Please go through this solution document and provide your input if this
> approach is okay or any drawback is there.
>
> Thanks & Regards
> Kejian Li
>
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[akashrn5]

Agree with Ajantha.

Regards
Akash

On Wed, Oct 14, 2020, 3:22 PM Ajantha Bhat <[hidden email]> wrote:

> Hi Kejian Li,
> Thanks for working on this.
>
> I see that this design and requirement is similar to what Nihal has
> discussed a few days ago.
>
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/DISCUSSION-Parallel-compaction-and-update-td100338.html
>
> So, Probably as Ravidra suggested for Nihal, maybe better to handle this by
> segment interface refactoring for this also.
>
> Thanks,
> Ajantha
>
> On Wed, Oct 14, 2020 at 2:46 PM Kejian Li <[hidden email]> wrote:
>
> > Dear community,
> >
> > This mail is regarding parallel insert and update. Currently we are not
> > supporting concurrent insert (or load data) and update because it may
> cause
> > data inconsistency or incorrect result.
> >
> > Now Carbon blocks update operation when insert operation is in progress
> by
> > throwing out concurrent operation exception directly. If there is an
> > executing insert operation that is very time consuming, then update
> > operation has to wait and sometimes this waiting time is very long.
> >
> > To come out with this problem, we are planning to support parallel insert
> > and update. And here I have proposed one of the solutions to implement
> this
> > feature.
> >
> > This is the design document:  Parallel_Insert_and_Update.pdf
> > <
> >
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/file/t495/Parallel_Insert_and_Update.pdf
> >
> >
> >
> > Please go through this solution document and provide your input if this
> > approach is okay or any drawback is there.
> >
> > Thanks & Regards
> > Kejian Li
> >
> >
> >
> >
> > --
> > Sent from:
> > http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
> >
>

[Kejian Li]

yes, I got it.  thanks.


--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Kejian Li]

yes, I got it. thanks.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[jacksanderojacks21313]

Pkr98 delivers a seamless mobile gaming experience, combining a user-friendly interface with reliable performance for all types of players. Whether you are seeking variety or ease of navigation, Pkr98 stands out as an accessible choice for quality digital entertainment. You can find the latest updates and gaming options by visiting the official website at Pkr.

[jacksanderojacks21313]

Okpkr is a digital platform designed for users interested in interactive mobile entertainment. By visiting the official website at Okpkr, you can explore the various features and services the platform provides. Okpkr is engineered to offer a responsive and straightforward interface for those looking to engage with its content on their mobile devices.

[jacksanderojacks2321424]

Start your next gaming adventure with a quick and easy jj55 game download. Experience seamless mobile entertainment and explore all the exciting features waiting for you on the platform.

[jacksanderojacks2321424]

Get ready for an effortless hh98 game download that brings exciting mobile entertainment straight to your Android device. Install the app today and dive right into the ultimate experience!

[Toronto11]

HH98 game download delivers a seamless mobile gaming experience, combining a user-friendly interface with reliable performance for all types of players. Whether you are seeking variety or ease of navigation, hh98 game download stands out as an accessible choice for quality digital entertainment. You can find the latest updates and gaming options by visiting the official website at Pkr.