[jira] [Created] (CARBONDATA-1582) 3. support StreamSinkProvider and append batch data to row format file

[Akash R Nilugal (Jira)]

QiangCai created CARBONDATA-1582:
------------------------------------

             Summary: 3. support StreamSinkProvider and append batch data to row format file
                 Key: CARBONDATA-1582
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1582
             Project: CarbonData
          Issue Type: Sub-task
          Components: data-load
            Reporter: QiangCai


 When data batch comes to carbon, carbon will check
whether the batch id of the data is already committed. If yes, skip this batch,
otherwise convert data fields and append to row format file directly. So, the
records are either flushed or not flushed, data consistency is ensured.

When writing to the row format file, carbon will convert the data fields the same way as for
columnar format, like global dictionary conversion, timestamp conversion, complex type
conversion, etc.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)