[
https://issues.apache.org/jira/browse/CARBONDATA-318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jacky Li updated CARBONDATA-318:
--------------------------------
Description:
Change SortDataRows into an External Sorter, it should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
1. addRow: insert rows into the sorter.
2. getIterator: will return an iterator that iterates on sorted rows, the sorted row could come from memory or files
was:
External Sorter should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
1. insertRow/insertRowBatch: will take an Iterator as input and insert rows from the iterator into the sorter.
2. getIterator: will return an iterator that iterates on sorted rows, the sorted row could come from memory or files
> Implement an ExternalSorter that makes maximum usage of memory while sorting
> ----------------------------------------------------------------------------
>
> Key: CARBONDATA-318
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-318> Project: CarbonData
> Issue Type: Sub-task
> Reporter: Jacky Li
> Fix For: 0.2.0-incubating
>
>
> Change SortDataRows into an External Sorter, it should sort in memory until it reach configured size, then spill to disk. It should provide following interface:
> 1. addRow: insert rows into the sorter.
> 2. getIterator: will return an iterator that iterates on sorted rows, the sorted row could come from memory or files
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
Locked
