Removing of kettle code from Carbondata

[ravipesala]

Hi All,

I guess it is time to remove the kettle flow from Carbondata loading. Now
there are two flows to load the data and becomes difficult to maintain the
code.Bug fixing or any feature implementation needs to be done in both the
places so it becomes difficult for developer to implement and test.

Please comment and vote on it.

--
Thanks & Regards,
Ravindra

[李寅威]

+1


well, I agree with Ravindra, As a better solution is born and it seems no backward compatibility issue in a data loading process.




------------------ 原始邮件 ------------------
发件人: "Ravindra Pesala";<[hidden email]>;
发送时间: 2017年3月11日(星期六) 上午10:21
收件人: "dev"<[hidden email]>;

主题: Removing of kettle code from Carbondata



Hi All,

I guess it is time to remove the kettle flow from Carbondata loading. Now
there are two flows to load the data and becomes difficult to maintain the
code.Bug fixing or any feature implementation needs to be done in both the
places so it becomes difficult for developer to implement and test.

Please comment and vote on it.

--
Thanks & Regards,
Ravindra

[Liang Chen]

Hi

Agree, +1.
The new data load(through spark) is quite stable and good performance, so i
agree to remove kettle flow for data loading.

Regards
Liang



2017-03-11 9:51 GMT+08:00 Ravindra Pesala <[hidden email]>:

> Hi All,
>
> I guess it is time to remove the kettle flow from Carbondata loading. Now
> there are two flows to load the data and becomes difficult to maintain the
> code.Bug fixing or any feature implementation needs to be done in both the
> places so it becomes difficult for developer to implement and test.
>
> Please comment and vote on it.
>
> --
> Thanks & Regards,
> Ravindra
>

--
Regards
Liang

[sraghunandan]

+1

On 11-Mar-2017 7:21 AM, "Ravindra Pesala" <[hidden email]> wrote:

Hi All,

I guess it is time to remove the kettle flow from Carbondata loading. Now
there are two flows to load the data and becomes difficult to maintain the
code.Bug fixing or any feature implementation needs to be done in both the
places so it becomes difficult for developer to implement and test.

Please comment and vote on it.

--
Thanks & Regards,
Ravindra

[cenyuhai]

+1




------------------ Original ------------------
From:  "Liang Chen";<[hidden email]>;
Date:  Sat, Mar 11, 2017 10:01 AM
To:  "dev"<[hidden email]>;

Subject:  Re: Removing of kettle code from Carbondata



Hi

Agree, +1.
The new data load(through spark) is quite stable and good performance, so i
agree to remove kettle flow for data loading.

Regards
Liang



2017-03-11 9:51 GMT+08:00 Ravindra Pesala <[hidden email]>:

> Hi All,
>
> I guess it is time to remove the kettle flow from Carbondata loading. Now
> there are two flows to load the data and becomes difficult to maintain the
> code.Bug fixing or any feature implementation needs to be done in both the
> places so it becomes difficult for developer to implement and test.
>
> Please comment and vote on it.
>
> --
> Thanks & Regards,
> Ravindra
>

--
Regards
Liang

[Aniket Adnaik]

+1 for Removing kettle
Agree, its confusing and cumbersome to maintain two different load paths.

Best Regards,
Aniket

On 10 Mar 2017 5:51 p.m., "Ravindra Pesala" <[hidden email]> wrote:

Hi All,

I guess it is time to remove the kettle flow from Carbondata loading. Now
there are two flows to load the data and becomes difficult to maintain the
code.Bug fixing or any feature implementation needs to be done in both the
places so it becomes difficult for developer to implement and test.

Please comment and vote on it.

--
Thanks & Regards,
Ravindra

[kumarvishal09]

+1
-Regards
Kumar Vishal

On Sat, Mar 11, 2017 at 10:33 AM, Aniket Adnaik <[hidden email]>
wrote:

> +1 for Removing kettle
> Agree, its confusing and cumbersome to maintain two different load paths.
>
> Best Regards,
> Aniket
>
> On 10 Mar 2017 5:51 p.m., "Ravindra Pesala" <[hidden email]> wrote:
>
> Hi All,
>
> I guess it is time to remove the kettle flow from Carbondata loading. Now
> there are two flows to load the data and becomes difficult to maintain the
> code.Bug fixing or any feature implementation needs to be done in both the
> places so it becomes difficult for developer to implement and test.
>
> Please comment and vote on it.
>
> --
> Thanks & Regards,
> Ravindra
>

[David CaiQiang]

+1

To avoid redundancy code,  better to do it after testing the new flow fully.

Regards
David QiangCai

[manishgupta88]

+1

Maintaining 2 flows is a cumbersome process. Better to remove.

Regards
Manish Gupta

On Mon, Mar 13, 2017 at 9:42 AM, QiangCai <[hidden email]> wrote:

> +1
>
> To avoid redundancy code,  better to do it after testing the new flow
> fully.
>
> Regards
> David QiangCai
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/Removing-of-
> kettle-code-from-Carbondata-tp8649p8724.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>

[ravipesala]

Hi David,

Thank you for your suggestion.
All known and major flows are tested and already it is the default flow in
current version.
Please let us know when you finish the new flow testing completely after
that we can initiate removing of kettle flow again.

Regards,
Ravindra.

On 13 March 2017 at 09:42, QiangCai <[hidden email]> wrote:

> +1
>
> To avoid redundancy code,  better to do it after testing the new flow
> fully.
>
> Regards
> David QiangCai
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/Removing-of-
> kettle-code-from-Carbondata-tp8649p8724.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>

--
Thanks & Regards,
Ravi

[Jacky Li]

+1 for removing the kettle code.

And how about the sorter implementation, currently, default property for off heap sort (ENABLE_UNSAFE_SORT) is false, how about making it to true and we can also remove heap sorter in the future after off heap sort is fully tested

Regards,
Jacky

> 在 2017年3月13日，下午1:05，Ravindra Pesala <[hidden email]> 写道：
>
> Hi David,
>
> Thank you for your suggestion.
> All known and major flows are tested and already it is the default flow in
> current version.
> Please let us know when you finish the new flow testing completely after
> that we can initiate removing of kettle flow again.
>
> Regards,
> Ravindra.
>
> On 13 March 2017 at 09:42, QiangCai <[hidden email]> wrote:
>
>> +1
>>
>> To avoid redundancy code,  better to do it after testing the new flow
>> fully.
>>
>> Regards
>> David QiangCai
>>
>>
>>
>> --
>> View this message in context: http://apache-carbondata-
>> mailing-list-archive.1130556.n5.nabble.com/Removing-of-
>> kettle-code-from-Carbondata-tp8649p8724.html
>> Sent from the Apache CarbonData Mailing List archive mailing list archive
>> at Nabble.com.
>>
>
>
> --
> Thanks & Regards,
> Ravi

[金铸]

+1


在 2017/3/11 9:51, Ravindra Pesala 写道:
> Hi All,
>
> I guess it is time to remove the kettle flow from Carbondata loading. Now
> there are two flows to load the data and becomes difficult to maintain the
> code.Bug fixing or any feature implementation needs to be done in both the
> places so it becomes difficult for developer to implement and test.
>
> Please comment and vote on it.
>

--
金铸
技术发展部（TDD）
东软集团股份有限公司
沈阳浑南新区新秀街2号东软软件园A2-105A
Postcode:110179
Tel: [phone]
Mobile：[phone]

 



---------------------------------------------------------------------------------------------------
Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s)
is intended only for the use of the intended recipient and may be confidential and/or privileged of
Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is
not the intended recipient, unauthorized use, forwarding, printing,  storing, disclosure or copying
is strictly prohibited, and may be unlawful.If you have received this communication in error,please
immediately notify the sender by return e-mail, and delete the original message and all copies from
your system. Thank you.
---------------------------------------------------------------------------------------------------