What is the transaction ability of CarbonData? Does it support the transaction like this.

Posted by 李书明 on
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/What-is-the-transaction-ability-of-CarbonData-Does-it-support-the-transaction-like-this-tp91939.html

Hi community!

I found CarbonData provides some transaction capabilities, However I have some questions about the detail transaction ability which I cannot find in the document.

There is a  problem like this:
  SessionA: insert overwrite tableA partition(dt=A) select * from x;
  SessionB: select * from tableA

For a common `Spark`(parquet) table, this situation may cause `FileNotFoundException` because `overwrite` op drops partitoin `dt=A` when SessionB is scanning this file.

However I tested `insert overwrite` of CarbonData table, it still dropped the tableA partition(dt=A).

I don’t know whether `CarbonData` source can solve this problem like this, the expected result of SessionB could be the data before `overwrite` or the complete data after `overwrite` instead of `FileNotFoundException`?