Hi All,
I guess it is time to remove the kettle flow from Carbondata loading. Now there are two flows to load the data and becomes difficult to maintain the code.Bug fixing or any feature implementation needs to be done in both the places so it becomes difficult for developer to implement and test. Please comment and vote on it. -- Thanks & Regards, Ravindra |
+1
well, I agree with Ravindra, As a better solution is born and it seems no backward compatibility issue in a data loading process. ------------------ 原始邮件 ------------------ 发件人: "Ravindra Pesala";<[hidden email]>; 发送时间: 2017年3月11日(星期六) 上午10:21 收件人: "dev"<[hidden email]>; 主题: Removing of kettle code from Carbondata Hi All, I guess it is time to remove the kettle flow from Carbondata loading. Now there are two flows to load the data and becomes difficult to maintain the code.Bug fixing or any feature implementation needs to be done in both the places so it becomes difficult for developer to implement and test. Please comment and vote on it. -- Thanks & Regards, Ravindra |
Administrator
|
In reply to this post by ravipesala
Hi
Agree, +1. The new data load(through spark) is quite stable and good performance, so i agree to remove kettle flow for data loading. Regards Liang 2017-03-11 9:51 GMT+08:00 Ravindra Pesala <[hidden email]>: > Hi All, > > I guess it is time to remove the kettle flow from Carbondata loading. Now > there are two flows to load the data and becomes difficult to maintain the > code.Bug fixing or any feature implementation needs to be done in both the > places so it becomes difficult for developer to implement and test. > > Please comment and vote on it. > > -- > Thanks & Regards, > Ravindra > -- Regards Liang |
In reply to this post by ravipesala
+1
On 11-Mar-2017 7:21 AM, "Ravindra Pesala" <[hidden email]> wrote: Hi All, I guess it is time to remove the kettle flow from Carbondata loading. Now there are two flows to load the data and becomes difficult to maintain the code.Bug fixing or any feature implementation needs to be done in both the places so it becomes difficult for developer to implement and test. Please comment and vote on it. -- Thanks & Regards, Ravindra |
In reply to this post by Liang Chen
+1
------------------ Original ------------------ From: "Liang Chen";<[hidden email]>; Date: Sat, Mar 11, 2017 10:01 AM To: "dev"<[hidden email]>; Subject: Re: Removing of kettle code from Carbondata Hi Agree, +1. The new data load(through spark) is quite stable and good performance, so i agree to remove kettle flow for data loading. Regards Liang 2017-03-11 9:51 GMT+08:00 Ravindra Pesala <[hidden email]>: > Hi All, > > I guess it is time to remove the kettle flow from Carbondata loading. Now > there are two flows to load the data and becomes difficult to maintain the > code.Bug fixing or any feature implementation needs to be done in both the > places so it becomes difficult for developer to implement and test. > > Please comment and vote on it. > > -- > Thanks & Regards, > Ravindra > -- Regards Liang |
In reply to this post by ravipesala
+1 for Removing kettle
Agree, its confusing and cumbersome to maintain two different load paths. Best Regards, Aniket On 10 Mar 2017 5:51 p.m., "Ravindra Pesala" <[hidden email]> wrote: Hi All, I guess it is time to remove the kettle flow from Carbondata loading. Now there are two flows to load the data and becomes difficult to maintain the code.Bug fixing or any feature implementation needs to be done in both the places so it becomes difficult for developer to implement and test. Please comment and vote on it. -- Thanks & Regards, Ravindra |
+1
-Regards Kumar Vishal On Sat, Mar 11, 2017 at 10:33 AM, Aniket Adnaik <[hidden email]> wrote: > +1 for Removing kettle > Agree, its confusing and cumbersome to maintain two different load paths. > > Best Regards, > Aniket > > On 10 Mar 2017 5:51 p.m., "Ravindra Pesala" <[hidden email]> wrote: > > Hi All, > > I guess it is time to remove the kettle flow from Carbondata loading. Now > there are two flows to load the data and becomes difficult to maintain the > code.Bug fixing or any feature implementation needs to be done in both the > places so it becomes difficult for developer to implement and test. > > Please comment and vote on it. > > -- > Thanks & Regards, > Ravindra >
kumar vishal
|
In reply to this post by ravipesala
+1
To avoid redundancy code, better to do it after testing the new flow fully. Regards David QiangCai
Best Regards
David Cai |
+1
Maintaining 2 flows is a cumbersome process. Better to remove. Regards Manish Gupta On Mon, Mar 13, 2017 at 9:42 AM, QiangCai <[hidden email]> wrote: > +1 > > To avoid redundancy code, better to do it after testing the new flow > fully. > > Regards > David QiangCai > > > > -- > View this message in context: http://apache-carbondata- > mailing-list-archive.1130556.n5.nabble.com/Removing-of- > kettle-code-from-Carbondata-tp8649p8724.html > Sent from the Apache CarbonData Mailing List archive mailing list archive > at Nabble.com. > |
In reply to this post by David CaiQiang
Hi David,
Thank you for your suggestion. All known and major flows are tested and already it is the default flow in current version. Please let us know when you finish the new flow testing completely after that we can initiate removing of kettle flow again. Regards, Ravindra. On 13 March 2017 at 09:42, QiangCai <[hidden email]> wrote: > +1 > > To avoid redundancy code, better to do it after testing the new flow > fully. > > Regards > David QiangCai > > > > -- > View this message in context: http://apache-carbondata- > mailing-list-archive.1130556.n5.nabble.com/Removing-of- > kettle-code-from-Carbondata-tp8649p8724.html > Sent from the Apache CarbonData Mailing List archive mailing list archive > at Nabble.com. > -- Thanks & Regards, Ravi |
+1 for removing the kettle code.
And how about the sorter implementation, currently, default property for off heap sort (ENABLE_UNSAFE_SORT) is false, how about making it to true and we can also remove heap sorter in the future after off heap sort is fully tested Regards, Jacky > 在 2017年3月13日,下午1:05,Ravindra Pesala <[hidden email]> 写道: > > Hi David, > > Thank you for your suggestion. > All known and major flows are tested and already it is the default flow in > current version. > Please let us know when you finish the new flow testing completely after > that we can initiate removing of kettle flow again. > > Regards, > Ravindra. > > On 13 March 2017 at 09:42, QiangCai <[hidden email]> wrote: > >> +1 >> >> To avoid redundancy code, better to do it after testing the new flow >> fully. >> >> Regards >> David QiangCai >> >> >> >> -- >> View this message in context: http://apache-carbondata- >> mailing-list-archive.1130556.n5.nabble.com/Removing-of- >> kettle-code-from-Carbondata-tp8649p8724.html >> Sent from the Apache CarbonData Mailing List archive mailing list archive >> at Nabble.com. >> > > > -- > Thanks & Regards, > Ravi |
In reply to this post by ravipesala
+1
在 2017/3/11 9:51, Ravindra Pesala 写道: > Hi All, > > I guess it is time to remove the kettle flow from Carbondata loading. Now > there are two flows to load the data and becomes difficult to maintain the > code.Bug fixing or any feature implementation needs to be done in both the > places so it becomes difficult for developer to implement and test. > > Please comment and vote on it. > -- 金铸 技术发展部(TDD) 东软集团股份有限公司 沈阳浑南新区新秀街2号东软软件园A2-105A Postcode:110179 Tel: (86 24)8366 2049 Mobile:13897999526 --------------------------------------------------------------------------------------------------- Confidentiality Notice: The information contained in this e-mail and any accompanying attachment(s) is intended only for the use of the intended recipient and may be confidential and/or privileged of Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader of this communication is not the intended recipient, unauthorized use, forwarding, printing, storing, disclosure or copying is strictly prohibited, and may be unlawful.If you have received this communication in error,please immediately notify the sender by return e-mail, and delete the original message and all copies from your system. Thank you. --------------------------------------------------------------------------------------------------- |
Free forum by Nabble | Edit this page |