[jira] [Created] (CARBONDATA-1522) 6. Loading aggregation tables for streaming data tables.

[Akash R Nilugal (Jira)]

Ravindra Pesala created CARBONDATA-1522:
-------------------------------------------

             Summary: 6. Loading aggregation tables for streaming data tables.
                 Key: CARBONDATA-1522
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-1522
             Project: CarbonData
          Issue Type: Sub-task
            Reporter: Ravindra Pesala


 we can finish the segment load after receives configurable amount of data and create a new segment to load new streaming data. While finishing the segment we can trigger the agg tables on it. So there would not be any agg tables on ongoing streaming segment but querying can be done on the streaming segment of actual table.
For example user configures stream_segment size as 1 GB, so for every 1 GB of stream data it receives it creates a new segment and finishes the current segment. While finishing the current segment we can trigger agg table loading and compaction of segments.
While querying of data we change the query plan to apply union of agg table and streaming segment of actual table to get the current data.




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)