[DISCUSSION] Propose to remove "support spark 1.5" from CarbonData 1.2.0 onwards

[Liang Chen-2]

Hi Dev

As you know, some key features only be supported by Spark 1.6 and Spark
2.1, for example : update and delete. so i propose to remove "support spark
1.5" from CarbonData 1.2.0 onwards, please discuss and vote.

Regards
Liang

[Jacky Li]

Currently, carbon integration with spark 1.5 and spark 1.6 almost share the same code. Is there any overhead or difficulties to maintain spark 1.5 integration code onward?

Regards,
Jacky

> 在 2017年7月8日，下午11:44，Liang Chen <[hidden email]> 写道：
>
> Hi Dev
>
> As you know, some key features only be supported by Spark 1.6 and Spark
> 2.1, for example : update and delete. so i propose to remove "support spark
> 1.5" from CarbonData 1.2.0 onwards, please discuss and vote.
>
> Regards
> Liang

[Erlu Chen]

I think it is ok to support spark1.5 without IUD currently.
If users upgrade spark version to spark2.1 or spark2.2 in the future, We can remove this if less user use spark1.5.
Is there any special reason which lead to this remove ?

Regards.
Chenerlu.

[Liang Chen-2]

Hi Jacky

You are right,  spark 1.5 integration code is almost same as 1.6, but still
has some difference.
Please find the below reasons for proposing remove spark 1.5 integration:
1.This proposal doesn't impact released carbondata version works with spark
1.5. What i am proposing that from 1.2.0 onwards, don't support Spark 1.5
integration , because 1.5 is very old version, would no people use spark
1.5 with carbondata.

2.Community doesn't arrange CI resource for spark 1.5 integration from
1.2.0 onwards.

3.From 1.2.0, carbondata community plans to provide binary release, spark
1.5 integration would not be considered.

In a word ,  from carbondata 1.2.0 onwards, for old spark 1.5 integration :
no people use and no CI to automatically ensure quality, so i propose to
remove it .

Regards
Liang



2017-07-09 1:24 GMT+08:00 Jacky Li <[hidden email]>:

> Currently, carbon integration with spark 1.5 and spark 1.6 almost share
> the same code. Is there any overhead or difficulties to maintain spark 1.5
> integration code onward?
>
> Regards,
> Jacky
>
> > 在 2017年7月8日，下午11:44，Liang Chen <[hidden email]> 写道：
> >
> > Hi Dev
> >
> > As you know, some key features only be supported by Spark 1.6 and Spark
> > 2.1, for example : update and delete. so i propose to remove "support
> spark
> > 1.5" from CarbonData 1.2.0 onwards, please discuss and vote.
> >
> > Regards
> > Liang
>
>
>
>