[DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

[DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

Liang Chen-2
Hi Dev

As you know, some key features only be supported by Spark 1.6 and Spark
2.1, for example : update and delete. so i propose to remove "support spark
1.5" from CarbonData 1.2.0 onwards, please discuss and vote.

Regards
Liang
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

Jacky Li
Currently, carbon integration with spark 1.5 and spark 1.6 almost share the same code. Is there any overhead or difficulties to maintain spark 1.5 integration code onward?

Regards,
Jacky

> 在 2017年7月8日,下午11:44,Liang Chen <[hidden email]> 写道:
>
> Hi Dev
>
> As you know, some key features only be supported by Spark 1.6 and Spark
> 2.1, for example : update and delete. so i propose to remove "support spark
> 1.5" from CarbonData 1.2.0 onwards, please discuss and vote.
>
> Regards
> Liang



Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

Erlu Chen
In reply to this post by Liang Chen-2
I think it is ok to support spark1.5 without IUD currently.
If users upgrade spark version to spark2.1 or spark2.2 in the future, We can remove this if less user use spark1.5.
Is there any special reason which lead to this remove ?

Regards.
Chenerlu.
Reply | Threaded
Open this post in threaded view
|

Re: [DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

Liang Chen-2
In reply to this post by Jacky Li
Hi Jacky

You are right,  spark 1.5 integration code is almost same as 1.6, but still
has some difference.
Please find the below reasons for proposing remove spark 1.5 integration:
1.This proposal doesn't impact released carbondata version works with spark
1.5. What i am proposing that from 1.2.0 onwards, don't support Spark 1.5
integration , because 1.5 is very old version, would no people use spark
1.5 with carbondata.

2.Community doesn't arrange CI resource for spark 1.5 integration from
1.2.0 onwards.

3.From 1.2.0, carbondata community plans to provide binary release, spark
1.5 integration would not be considered.

In a word ,  from carbondata 1.2.0 onwards, for old spark 1.5 integration :
no people use and no CI to automatically ensure quality, so i propose to
remove it .

Regards
Liang



2017-07-09 1:24 GMT+08:00 Jacky Li <[hidden email]>:

> Currently, carbon integration with spark 1.5 and spark 1.6 almost share
> the same code. Is there any overhead or difficulties to maintain spark 1.5
> integration code onward?
>
> Regards,
> Jacky
>
> > 在 2017年7月8日,下午11:44,Liang Chen <[hidden email]> 写道:
> >
> > Hi Dev
> >
> > As you know, some key features only be supported by Spark 1.6 and Spark
> > 2.1, for example : update and delete. so i propose to remove "support
> spark
> > 1.5" from CarbonData 1.2.0 onwards, please discuss and vote.
> >
> > Regards
> > Liang
>
>
>
>