Re: SPARK update error

Posted by Erlu Chen on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/SPARK-update-error-tp13144p13211.html

Hi, sunerhan

As I know, Carbon support upward compatible，that means if we create carbon table in spark1.6 + carbon1.1.0,we can get access this created table in spark2.1 + carbon1.1.0.

Actually, Carbon will do schema path check in spark 2.1 + carbon1.1.0. Besides, Carbon use different CarbonDatasourceHadoopRelation for integrating spark1.6 and spark2.1. This part of process logic has been refactored.

Carbon can't identify your carbon table created in spark2.1 + carbon1.1.0 while you query it in spark1.6 + carbon1.1.0, it may be treated as hive table, and if the path is not exists or the schema file is incorrect, then error will happen.

Regards.
Chenerlu