after load data using SaveMode.Overwrite, query through beeline return all null field

Posted by 喜之郎 on May 18, 2018; 3:46am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/after-load-data-using-SaveMode-Overwrite-query-through-beeline-return-all-null-field-tp49658.html

hi dev.
carbon version :1.3.1
spark version:2.2.1
1) First I create a carbon table through beeline.
2) Then I use spark-submit and dataframe load data to carbon. Query is OK。
3) Then I use spark-submit and dataframe load data to carbon again, but query through thriftserver return all NULL field.
4) I restart thriftserver, and query through thriftserver return correct result.


This problem happens everytime. I think this problem may is about CarbonThriftServer.
is there anybody encountered this problem? can anybody give some advice,thanks.


you can use this code to reproduce problem.
===================



val carbonTableName = args(1)

val warehouse = new File("./warehouse").getCanonicalPath
val metastore = new File("./metastore").getCanonicalPath
val spark = SparkSession
  .builder()
  .appName("StreamExample")
  .config("spark.sql.warehouse.dir", warehouse)
  .getOrCreateCarbonSession(warehouse, metastore)
val df2 = spark.read.parquet(path)
df2.write
  .format("carbondata")
  .option("tableName", carbonTableName)
  .option("compress", "true")
  .option("tempCSV", "false")
  .mode(SaveMode.Overwrite)
  .save()=============