[
https://issues.apache.org/jira/browse/CARBONDATA-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16093548#comment-16093548 ]
kumar vishal commented on CARBONDATA-1318:
------------------------------------------
Reason for this issue is simultaneous running of multiple table data loading. In this case when Unsafe memory is full then in sort step there is no memory left and hence data loading is failing. To handle this scenario we can have two manager
1. Working memory(Temp memory for operation like sorting, query processing)
2. sorted data storage memory.
Sorting will be done in working memory. After sorting data will be copied to sorted data storage memory.
so working memory will be always free.
Working memory can be also used during query and storing column pages
> Concurrent data loading is failing
> ----------------------------------
>
> Key: CARBONDATA-1318
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-1318> Project: CarbonData
> Issue Type: Bug
> Reporter: kumar vishal
> Assignee: kumar vishal
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Councurrent data loading is failing as unsafememory manager doesnot have space and it is throwing outofmemory exception
> When Job is cancelled unsafe memory manager is not clearing the resource
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
Locked
