Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

xm_zzc
Hi all:
  when updating data, if 'carbon.update.persist.enable'='true', the dataset
in the method 'ProjectForUpdateCommand.processData' will be persisted, but
now it's storage level is MEMORY_AND_DISK, should it be set by user or not?
  if yes, I can implement it, and add an option such as
'carbon.update.storage.level' to configurate.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

xm_zzc
Hi all, any suggestion about this?



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

xuchuanyin-2
I remembered a similar option had been added in loading data with global sort scope.

So I suggest to check whether there are some other similar problems.



On 09/05/2017 13:15, xm_zzc wrote:
Hi all, any suggestion about this?



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

xm_zzc
I have searched and there was no other similar problem except this.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

ravipesala
Hi,

I don't see any problem in adding the options although MEMORY_AND_DISK is a
preferable option. You can keep it as a developer options only and no need
to expose to the user.

Regards,
Ravindra.

On 5 September 2017 at 11:15, xm_zzc <[hidden email]> wrote:

> I have searched and there was no other similar problem except this.
>
>
>
> --
> Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.
> n5.nabble.com/
>



--
Thanks & Regards,
Ravi
Reply | Threaded
Open this post in threaded view
|

Re: Add an option such as 'carbon.update.storage.level' to configurate the storage level when updating data with 'carbon.update.persist.enable'='true'

xm_zzc
ok, I will raise a pr to implement this.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/