[Discuss] Removing search mode

[Jacky Li]

Hi,

Search mode was introduced as an alpha feature in spark integration during CarbonData 1.4.0, its goal was to increase the concurrent query performance. However, after several experiments, it only show very little improvement. The main reason is that SparkSQL is designed based on DAG/MR compute model, thus it is not good at low latency high concurrent queries. A possibly better approach is to provide such query using presto-carbon integration. So, I propose to make this alpha feature obsolete and remove from master code base to reduce the complexity of the overall project.

Regards,
Jacky

[akashrn5]

+1
yes, after search mode implementation we didnt get much advantage as
expected and simply code will be complex, i agree with likun.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Liang Chen]

Hi

+1, but one suggestion,  in the future we can first try these alpha features
in the separate branch . once it is confirmed, then merge into master.

Regards
Liang


akashrn5 wrote
> +1
> yes, after search mode implementation we didnt get much advantage as
> expected and simply code will be complex, i agree with likun.
>
>
>
> --
> Sent from:
> http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/





--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[xuchuanyin]

+1

Q1: When will we start and finish the optimization in carbon-presto
integration? Any plan for this?

Another question:
 Q2: Is it possible to use carbon reader to implement the similar function
of search mode?




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/

[Jacky Li]

Currenlty, presto-carbon is missing the direct vector fill implementation which is implemented for spark in latest master branch.
I think we can plan this feature in next version.

Since presto’s architecture is similar to MPP, so it will execute in a pipeline manner, which it is good for interactive query.

Regards,
Jacky

> 在 2018年11月6日，下午8:56，xuchuanyin <[hidden email]> 写道：
>
> +1
>
> Q1: When will we start and finish the optimization in carbon-presto
> integration? Any plan for this?
>
> Another question:
> Q2: Is it possible to use carbon reader to implement the similar function
> of search mode?
>
>
>
>
> --
> Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
>

[xubo245]

+1,

There are some random error in CI recently. and the performance only has a
little improvement between search mode and non search mode, including filter
query.



--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/