carbondata1.5 compatible with spark2.3.1

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

carbondata1.5 compatible with spark2.3.1

付涛
it seems that carbondata 1.5 will using spark2.3.2,so the spark2.3.1will be
deprecated。however,our online circumstance is based on spark2.3.1,and now we
are using carbondata1.5 master to test, and we find the effect of query is
good。we hope that carbondata1.5 shoud be compatibled with spark2.3.1,so that
our circumstance will not be affected。



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: carbondata1.5 compatible with spark2.3.1

xm_zzc
Hi:
   CarbonData community decide to support spark 2.3.2 only, because spark
2.3.1 has some critical issues. If you need to integrate spark 2.3.1 ,
please contact me on mailling list and I will told you how to do.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: carbondata1.5 compatible with spark2.3.1

sujith chacko
In reply to this post by 付涛
Hi ,

Carbon is compatible with both versions spark 2.3.1 and spark 2.3.2 .
but it will be always better to use more stable versions for online
deployment. Spark 2.3.2 has many blockers and data inconsistency issues
fixed including few security bugs.

Thanks,
Sujith

On Sun, 30 Sep 2018 at 1:46 PM, 付涛 <[hidden email]> wrote:

> it seems that carbondata 1.5 will using spark2.3.2,so the spark2.3.1will be
> deprecated。however,our online circumstance is based on spark2.3.1,and now
> we
> are using carbondata1.5 master to test, and we find the effect of query is
> good。we hope that carbondata1.5 shoud be compatibled with spark2.3.1,so
> that
> our circumstance will not be affected。
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Reply | Threaded
Open this post in threaded view
|

Re: carbondata1.5 compatible with spark2.3.1

xm_zzc
Hi Sujith:
  Currently it can't compile with Spark 2.3.1 after merging PR#2779, because
Spark 2.3.2 has some interface changes, please see
https://github.com/apache/carbondata/pull/2779
<https://github.com/apache/carbondata/pull/2779>  



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/