Re: [DISCUSSION] support user specified segment reading for query

Posted by sraghunandan on Oct 09, 2017; 12:54am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/DISCUSSION-support-user-specified-segment-reading-for-query-tp23499p23585.html

Query hint is a better option.
Some down side in using this approach:
User will have to modify all his query to accommodate this.
This won't be session scope but rather query scope
On Mon, 9 Oct 2017 at 2:36 AM, jarray <[hidden email]> wrote:

> same opinion with @Ravindra，use query hint is better approach ，the syntax
> is easy to use
>
>
>
>
>
>
> On 10/05/2017 17:10, Rahul Kumar wrote:
> @Ravindra Thanks. Using query hint may be better approach.
>
> I suggest following syntax :
>
> *select * from t1 [in SEGMENTS(1,3,5)]; *
>
>          Thanks and Regards
>
> *   Rahul Kumar     *
>
>
>
> On Thu, Oct 5, 2017 at 1:04 PM, Ravindra Pesala <[hidden email]>
> wrote:
>
> > Hi,
> >
> > Instead of using SET command to use for segments why don't you use QUERY
> > HINT . Using query hint we can mention the segments inside the query
> itself
> > as a hint.
> >
> > For example  SELECT /*+SEGMENTS(1,3,5) */ from t1.
> >
> > By using the above custom hint we can query from selected segments only,
> > This concept is supported in Spark also and this concept will be helpful
> in
> > our any future optimizations
> >
> > Regards,
> > Ravindra.
> >
> > On 5 October 2017 at 12:22, Rahul Kumar <[hidden email]> wrote:
> >
> > > @Jacky please find the reply of your doubts as follow :
> > >
> > >
> > > *1.    If user uses following command in two different beeline session,
> > > will there be problem due to multithreading?       SET
> > > carbon.input.segments.default.*
> > >
> > > *carbontable=1,3,5;       select * from carbontable;       SET
> > > carbon.input.segments.default.**carbontable=*;*
> > >
> > > *Ans: *In case of multithreading ,yes there will be problem.
> > >
> > >       So threadSet() can be use to set the same property in multithread
> > > mode.
> > > *      Folowing syntax can be used to set segment ids for multithread
> > mode*
> > > :
> > >    Syntax : CarbonSession.threadSet(“carbon.input.segments.<
> > > databese_name>.<table_name>”,”<list
> > > of segment ids>”)
> > >    e.g =>*future{*
> > >
> > > * CarbonSession.threadSet(“**carbon.input.segments.
> > > default.carbontable”,”1,3,5”)*
> > >
> > > * sparkSession.sql(“select * from carbontable”).show*
> > >
> > > * CarbonSession.threadSet(“carbon.input.segments.
> > > default.carbontable”,”*”)*
> > >
> > > * }*
> > >
> > > *Above will override the property at thread level. So property will be
> > set
> > > for each thread .*
> > >
> > >
> > > *2.   The RESET command is not clear, why this is needed? It seems SET
> > > carbon.input.segments.default.**carbontable=* is enough, right? and
> what
> > > parameter it has?*
> > >
> > > *Ans:* RESET command doesn't take any parameter. RESET is already
> > > implemented behavior which resets all the properties to their default
> > > value.So simillarly RESET query will set the above property also to its
> > > default value.
> > >
> > >           Thanks and Regards
> > >
> > > *   Rahul Kumar     *
> > >
> > >
> > >
> > > On Wed, Oct 4, 2017 at 7:21 PM, Jacky Li <[hidden email]> wrote:
> > >
> > > > I have 2 doubts:
> > > > 1. If user uses following command in two different beeline session,
> > will
> > > > there be problem due to multithreading?
> > > > SET carbon.input.segments.default.carbontable=1,3,5;
> > > > select * from carbontable;
> > > > SET carbon.input.segments.default.carbontable=*;
> > > >
> > > >
> > > > 2. The RESET command is not clear, why this is needed? It seems SET
> > > > carbon.input.segments.default.carbontable=* is enough, right? and
> what
> > > > parameter it has?
> > > >
> > > > Regards,
> > > > Jacky
> > > >
> > > > > 在 2017年10月4日，上午12:42，Rahul Kumar <[hidden email]> 写道：
> > > > >
> > > > > <segmentReading.odt>
> > > >
> > > >
> > >
> >
> >
> >
> > --
> > Thanks & Regards,
> > Ravi
> >
>