[jira] [Comment Edited] (CARBONDATA-1022) Get errors while do "select" query in Spark-shell

[Akash R Nilugal (Jira)]

    [ https://issues.apache.org/jira/browse/CARBONDATA-1022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15999633#comment-15999633 ]

Liang Chen edited comment on CARBONDATA-1022 at 5/7/17 12:26 AM:
-----------------------------------------------------------------

Version : download "spark-1.6.1-bin-hadoop2.6" + CarbonData master jar(carbondata_2.10-1.1.0-SNAPSHOT-shade-hadoop2.2.0.jar)

Reproduced script as below:
$ ##set SPARK_HOME = spark1.6...##

$ carbondata_jar=./lib/$(ls -1 lib |grep "^carbondata_.*\.jar$")
$ ./bin/spark-shell --master local --jars ${carbondata_jar}
$ import org.apache.spark.sql.CarbonContext
$ val cc = new CarbonContext(sc, "./carbondata/store", "./carbondata/metastorelocation")
$ cc.sql("CREATE TABLE connectdemo1 (name String, gender String, province String, singler String, age Int) STORED BY 'org.apache.carbondata.format'")
$ cc.sql("LOAD DATA LOCAL INPATH '/Users/apple/DEMO/3Mtestdata.csv' INTO TABLE connectdemo1")
$ scala> cc.sql("select * from connectdemo1").show
17/05/07 08:20:03 ERROR SegmentUpdateStatusManager: pool-157-thread-1 Invalid tuple id op2.6/carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1494116377555/0
+------+------+--------+-------+---+
|  name|gender|province|singler|age|
+------+------+--------+-------+---+
|Abbott|Female|      AB|  false| 39|
|Abbott|Female|      AB|  false| 62|
|Abbott|Female|      AB|  false| 62|
|Abbott|Female|      AB|  false| 45|
|Abbott|Female|      AB|  false| 40|
|Abbott|Female|      AB|  false| 32|
|Abbott|Female|      AB|  false| 29|
|Abbott|Female|      AB|  false| 38|


was (Author: chenliang613):
Version : download "spark-1.6.1-bin-hadoop2.6" + CarbonData master jar(carbondata_2.10-1.1.0-SNAPSHOT-shade-hadoop2.2.0.jar)

Reproduced script as below:
$ import org.apache.spark.sql.CarbonContext
$ val cc = new CarbonContext(sc, "./carbondata/store", "./carbondata/metastorelocation")
$ cc.sql("CREATE TABLE connectdemo1 (name String, gender String, province String, singler String, age Int) STORED BY 'org.apache.carbondata.format'")
$ cc.sql("LOAD DATA LOCAL INPATH '/Users/apple/DEMO/3Mtestdata.csv' INTO TABLE connectdemo1")
$ scala> cc.sql("select * from connectdemo1").show
17/05/07 08:20:03 ERROR SegmentUpdateStatusManager: pool-157-thread-1 Invalid tuple id op2.6/carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1494116377555/0
+------+------+--------+-------+---+
|  name|gender|province|singler|age|
+------+------+--------+-------+---+
|Abbott|Female|      AB|  false| 39|
|Abbott|Female|      AB|  false| 62|
|Abbott|Female|      AB|  false| 62|
|Abbott|Female|      AB|  false| 45|
|Abbott|Female|      AB|  false| 40|
|Abbott|Female|      AB|  false| 32|
|Abbott|Female|      AB|  false| 29|
|Abbott|Female|      AB|  false| 38|

> Get errors while do "select" query in Spark-shell
> -------------------------------------------------
>
>                 Key: CARBONDATA-1022
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1022
>             Project: CarbonData
>          Issue Type: Bug
>          Components: sql
>            Reporter: Liang Chen
>
> Spark1.6's spark-shell, error info as below :
> scala> cc.sql("select * from connectdemo1").show
> 17/05/04 23:06:36 ERROR SegmentUpdateStatusManager: pool-178-thread-1 Invalid tuple id /carbondata/store/default/connectdemo1/Fact/0/0/0-0_batchno0-0-1493910126468/0

--
This message was sent by Atlassian JIRA
(v6.3.15#6346)