Re: [DISCUSSION] support user specified segment reading for query

Posted by ravipesala on Oct 05, 2017; 7:34am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-support-user-specified-segment-reading-for-query-tp23499p23522.html

Hi,

Instead of using SET command to use for segments why don't you use QUERY
HINT . Using query hint we can mention the segments inside the query itself
as a hint.

For example  SELECT /*+SEGMENTS(1,3,5) */ from t1.

By using the above custom hint we can query from selected segments only,
This concept is supported in Spark also and this concept will be helpful in
our any future optimizations

Regards,
Ravindra.

On 5 October 2017 at 12:22, Rahul Kumar <[hidden email]> wrote:

> @Jacky please find the reply of your doubts as follow :
>
>
> *1.    If user uses following command in two different beeline session,
> will there be problem due to multithreading?       SET
> carbon.input.segments.default.*
>
> *carbontable=1,3,5;       select * from carbontable;       SET
> carbon.input.segments.default.**carbontable=*;*
>
> *Ans: *In case of multithreading ,yes there will be problem.
>
>       So threadSet() can be use to set the same property in multithread
> mode.
> *      Folowing syntax can be used to set segment ids for multithread mode*
> :
>    Syntax : CarbonSession.threadSet(“carbon.input.segments.<
> databese_name>.<table_name>”,”<list
> of segment ids>”)
>    e.g =>*future{*
>
> * CarbonSession.threadSet(“**carbon.input.segments.
> default.carbontable”,”1,3,5”)*
>
> * sparkSession.sql(“select * from carbontable”).show*
>
> * CarbonSession.threadSet(“carbon.input.segments.
> default.carbontable”,”*”)*
>
> * }*
>
> *Above will override the property at thread level. So property will be set
> for each thread .*
>
>
> *2.   The RESET command is not clear, why this is needed? It seems SET
> carbon.input.segments.default.**carbontable=* is enough, right? and what
> parameter it has?*
>
> *Ans:* RESET command doesn't take any parameter. RESET is already
> implemented behavior which resets all the properties to their default
> value.So simillarly RESET query will set the above property also to its
> default value.
>
>           Thanks and Regards
>
> *   Rahul Kumar     *
>
>
>
> On Wed, Oct 4, 2017 at 7:21 PM, Jacky Li <[hidden email]> wrote:
>
> > I have 2 doubts:
> > 1. If user uses following command in two different beeline session, will
> > there be problem due to multithreading?
> > SET carbon.input.segments.default.carbontable=1,3,5;
> > select * from carbontable;
> > SET carbon.input.segments.default.carbontable=*;
> >
> >
> > 2. The RESET command is not clear, why this is needed? It seems SET
> > carbon.input.segments.default.carbontable=* is enough, right? and what
> > parameter it has?
> >
> > Regards,
> > Jacky
> >
> > > 在 2017年10月4日，上午12:42，Rahul Kumar <[hidden email]> 写道：
> > >
> > > <segmentReading.odt>
> >
> >
>

--
Thanks & Regards,
Ravi