carbon thrift server for spark 2.0 showing unusual behaviour

[anubhavtarar]

carbon thrift server is not parsing carbon query randomly with spark 2

here are the logs

: jdbc:hive2://localhost:10000> CREATE TABLE Bug221755915(int int)USING org.apache.spark.sql.CarbonSource;
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.216 seconds)

can anyone help?

[David CaiQiang]

Can you show the JDBCServer startup script?

[anubhavtarar]

here is the script ./bin/spark-submit --conf
spark.sql.hive.thriftServer.singleSession=true --class
org.apache.carbondata.spark.thriftserver.CarbonThriftServer
/opt/spark-2.0.0-bin-hadoop2.7/carbonlib/carbondata_2.11-1.0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar
hdfs://localhost:54310/opt/carbonStore

it works fine only 1 out of 10 times

On Wed, Jan 4, 2017 at 8:28 AM, QiangCai <[hidden email]> wrote:

> Can you show the JDBCServer startup script?
>
>
>
> --
> View this message in context: http://apache-carbondata-
> mailing-list-archive.1130556.n5.nabble.com/carbon-thrift-
> server-for-spark-2-0-showing-unusual-behaviour-tp5384p5419.html
> Sent from the Apache CarbonData Mailing List archive mailing list archive
> at Nabble.com.
>

--
Thanks and Regards

*   Anubhav Tarar     *


* Software Consultant*
      *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
       LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
<https://twitter.com/RahulKu71223673>    fb <[hidden email]>
          mob : [phone]

[ravipesala]

Hi,

I did not understand the issue, what is the error it throws?

On 4 January 2017 at 10:03, Anubhav Tarar <[hidden email]> wrote:

> here is the script ./bin/spark-submit --conf
> spark.sql.hive.thriftServer.singleSession=true --class
> org.apache.carbondata.spark.thriftserver.CarbonThriftServer
> /opt/spark-2.0.0-bin-hadoop2.7/carbonlib/carbondata_2.11-1.
> 0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar
> hdfs://localhost:54310/opt/carbonStore
>
> it works fine only 1 out of 10 times
>
> On Wed, Jan 4, 2017 at 8:28 AM, QiangCai <[hidden email]> wrote:
>
> > Can you show the JDBCServer startup script?
> >
> >
> >
> > --
> > View this message in context: http://apache-carbondata-
> > mailing-list-archive.1130556.n5.nabble.com/carbon-thrift-
> > server-for-spark-2-0-showing-unusual-behaviour-tp5384p5419.html
> > Sent from the Apache CarbonData Mailing List archive mailing list archive
> > at Nabble.com.
> >
>
>
>
> --
> Thanks and Regards
>
> *   Anubhav Tarar     *
>
>
> * Software Consultant*
>       *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
>        LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
> <https://twitter.com/RahulKu71223673>    fb <[hidden email]>
>           mob : [phone]
>

--
Thanks & Regards,
Ravi

[anubhavtarar]

see these logs
jdbc:hive2://localhost:10000>  CREATE TABLE Bug2217559156(int int)USING
org.apache.spark.sql.CarbonSource;
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (1.809 seconds)
it does not give any exception in any query most of the times but sometimes
it works


On Wed, Jan 4, 2017 at 12:54 PM, Ravindra Pesala <[hidden email]>
wrote:

> Hi,
>
> I did not understand the issue, what is the error it throws?
>
> On 4 January 2017 at 10:03, Anubhav Tarar <[hidden email]>
> wrote:
>
> > here is the script ./bin/spark-submit --conf
> > spark.sql.hive.thriftServer.singleSession=true --class
> > org.apache.carbondata.spark.thriftserver.CarbonThriftServer
> > /opt/spark-2.0.0-bin-hadoop2.7/carbonlib/carbondata_2.11-1.
> > 0.0-incubating-SNAPSHOT-shade-hadoop2.2.0.jar
> > hdfs://localhost:54310/opt/carbonStore
> >
> > it works fine only 1 out of 10 times
> >
> > On Wed, Jan 4, 2017 at 8:28 AM, QiangCai <[hidden email]> wrote:
> >
> > > Can you show the JDBCServer startup script?
> > >
> > >
> > >
> > > --
> > > View this message in context: http://apache-carbondata-
> > > mailing-list-archive.1130556.n5.nabble.com/carbon-thrift-
> > > server-for-spark-2-0-showing-unusual-behaviour-tp5384p5419.html
> > > Sent from the Apache CarbonData Mailing List archive mailing list
> archive
> > > at Nabble.com.
> > >
> >
> >
> >
> > --
> > Thanks and Regards
> >
> > *   Anubhav Tarar     *
> >
> >
> > * Software Consultant*
> >       *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
> >        LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
> > <https://twitter.com/RahulKu71223673>    fb <[hidden email]>
> >           mob : [phone]
> >
>
>
>
> --
> Thanks & Regards,
> Ravi
>

--
Thanks and Regards

*   Anubhav Tarar     *


* Software Consultant*
      *Knoldus Software LLP <http://www.knoldus.com/home.knol>       *
       LinkedIn <http://in.linkedin.com/in/rahulforallp>     Twitter
<https://twitter.com/RahulKu71223673>    fb <[hidden email]>
          mob : [phone]

[anubhavtarar]

have a look at these logs

CREATE TABLE Bug212(int string)USING org.apache.spark.sql.CarbonSource OPTIONS("bucketnumber"="1", "bucketcolumns"="String","tableName"="t100");

Error: org.apache.carbondata.spark.exception.MalformedCarbonCommandException: Table default.t 100 can not be created without key columns. Please use DICTIONARY_INCLUDE or DICTIONARY_EXCLUDE to set at least one key column if all specified columns are numeric types (state=,code=0)

2 minutes later

 CREATE TABLE Bug211(int int)USING org.apache.spark.sql.CarbonSource OPTIONS("bucketnumber"="1", "bucketcolumns"="String","tableName"="t 100");

it works

+---------+--+
| Result  |
+---------+--+
+---------+--+


it is blocking for spark2 testing

[David CaiQiang]

1. column name is "int"?  type is "String"?  

the bucketcolumns  should use column name.
better to try another column name.

2. create name should be same with option name .

[anubhavtarar]

see this

CREATE TABLE Bug2122167(bug INT)USING org.apache.spark.sql.CarbonSource OPTIONS("bucketnumber"="1", "bucketcolumns"="int","table"="t100");
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.212 seconds)

i got the result

then see this

CREATE TABLE Bug222167(bug INT)USING org.apache.spark.sql.CarbonSource OPTIONS("bucketnumber"="1", "bucketcolumns"="int","tablee"="t 100");
+---------+--+
| Result  |
+---------+--+
+---------+--+
No rows selected (0.212 seconds)

here table is writeen as tablee my point is that for the first time when i run query from beeline first time it is validating all the checks in carbon data but second time same query fails