Version upgrade for Presto Integration to 0.186

[bhavya411]

Hi All,

Presto 0.186 version has as lot of improvements that will increase the
performance and improve the reliability. Some of the major issues and
improvements are listed below.


   - Fix excessive GC overhead caused by map to map cast.
   - Fix issue that may cause queries containing expensive functions, such
   as regular expressions, to continue using CPU resources even after they are
   killed.
   - Fix performance issue caused by redundant casts
   - Fix leak in running query counter for failed queries. The counter
   would increment but never decrement for queries that failed before starting.
   - Reduce memory usage when building data of VARCHAR or VARBINARY types.
   - Estimate memory usage for GROUP BY more precisely to avoid out of
   memory errors.
   - Add Spill to Disk <https://prestodb.io/docs/current/admin/spill.html>
   for joins.

Currently the Presto version that we are using in Carbondata is 0.166 , I
would like to suggest to upgrade it to 0.186. Please let me know what the
group thinks about it.


Regards

Bhavya

[sraghunandan]

Any backward incompatibilities introduced?
+1 for the upgrade
On Thu, 2 Nov 2017 at 12:18 PM, Bhavya Aggarwal <[hidden email]> wrote:

> Hi All,
>
> Presto 0.186 version has as lot of improvements that will increase the
> performance and improve the reliability. Some of the major issues and
> improvements are listed below.
>
>
>    - Fix excessive GC overhead caused by map to map cast.
>    - Fix issue that may cause queries containing expensive functions, such
>    as regular expressions, to continue using CPU resources even after they
> are
>    killed.
>    - Fix performance issue caused by redundant casts
>    - Fix leak in running query counter for failed queries. The counter
>    would increment but never decrement for queries that failed before
> starting.
>    - Reduce memory usage when building data of VARCHAR or VARBINARY types.
>    - Estimate memory usage for GROUP BY more precisely to avoid out of
>    memory errors.
>    - Add Spill to Disk <https://prestodb.io/docs/current/admin/spill.html>
>    for joins.
>
> Currently the Presto version that we are using in Carbondata is 0.166 , I
> would like to suggest to upgrade it to 0.186. Please let me know what the
> group thinks about it.
>
>
> Regards
>
> Bhavya
>

[phalodi]

+1. SPI are backward incompatible so make  sure if you use the SPI for
presto integration you need to change following things :


   - Remove owner from ConnectorTableMetadata.
   - Replace the generic getServices() method in Plugin with specific
   methods such as getConnectorFactories(), getTypes(), etc. Dependencies
   like TypeManager are now provided directly rather than being injected
   into Plugin.
   - Add first-class support for functions in the SPI. This replaces the
   old FunctionFactory interface. Plugins can return a list of classes from
   the getFunctions() method:
      - Scalar functions are methods or classes annotated with
      @ScalarFunction.
      - Aggregation functions are methods or classes annotated with
      @AggregationFunction.
      - Window functions are an implementation of WindowFunction. Most
      implementations should be a subclass of RankingWindowFunction or
      ValueWindowFunction.

-Sandeep


On Thu, Nov 2, 2017 at 6:53 PM, Raghunandan S <
[hidden email]> wrote:

> Any backward incompatibilities introduced?
> +1 for the upgrade
> On Thu, 2 Nov 2017 at 12:18 PM, Bhavya Aggarwal <[hidden email]>
> wrote:
>
> > Hi All,
> >
> > Presto 0.186 version has as lot of improvements that will increase the
> > performance and improve the reliability. Some of the major issues and
> > improvements are listed below.
> >
> >
> >    - Fix excessive GC overhead caused by map to map cast.
> >    - Fix issue that may cause queries containing expensive functions,
> such
> >    as regular expressions, to continue using CPU resources even after
> they
> > are
> >    killed.
> >    - Fix performance issue caused by redundant casts
> >    - Fix leak in running query counter for failed queries. The counter
> >    would increment but never decrement for queries that failed before
> > starting.
> >    - Reduce memory usage when building data of VARCHAR or VARBINARY
> types.
> >    - Estimate memory usage for GROUP BY more precisely to avoid out of
> >    memory errors.
> >    - Add Spill to Disk <https://prestodb.io/docs/
> current/admin/spill.html>
> >    for joins.
> >
> > Currently the Presto version that we are using in Carbondata is 0.166 , I
> > would like to suggest to upgrade it to 0.186. Please let me know what the
> > group thinks about it.
> >
> >
> > Regards
> >
> > Bhavya
> >
>

[Liang Chen]

+1

Can you raise one PR for this.

Regards
Liang


bhavya411 wrote

> Hi All,
>
> Presto 0.186 version has as lot of improvements that will increase the
> performance and improve the reliability. Some of the major issues and
> improvements are listed below.
>
>
>    - Fix excessive GC overhead caused by map to map cast.
>    - Fix issue that may cause queries containing expensive functions, such
>    as regular expressions, to continue using CPU resources even after they
> are
>    killed.
>    - Fix performance issue caused by redundant casts
>    - Fix leak in running query counter for failed queries. The counter
>    would increment but never decrement for queries that failed before
> starting.
>    - Reduce memory usage when building data of VARCHAR or VARBINARY types.
>    - Estimate memory usage for GROUP BY more precisely to avoid out of
>    memory errors.
>    - Add Spill to Disk
> &lt;https://prestodb.io/docs/current/admin/spill.html&gt;
>    for joins.
>
> Currently the Presto version that we are using in Carbondata is 0.166 , I
> would like to suggest to upgrade it to 0.186. Please let me know what the
> group thinks about it.
>
>
> Regards
>
> Bhavya

--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/