[jira] [Created] (CARBONDATA-2927) Multiple issue fixes for varchar column and complex columns that grows more than 2MB

[Akash R Nilugal (Jira)]

Ajantha Bhat created CARBONDATA-2927:
----------------------------------------

             Summary: Multiple issue fixes for varchar column and complex columns that grows more than 2MB
                 Key: CARBONDATA-2927
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2927
             Project: CarbonData
          Issue Type: Bug
            Reporter: Ajantha Bhat
            Assignee: Ajantha Bhat


*Fixed:*
 *1. varchar data length is more than 2MB, buffer overflow exception (thread local row buffer)*

*root* casue*: thread* loaclbuffer *was hardcoded with 2MB.*

*solution: grow dynamically based on the row size.*


 *2. read data from carbon file having one row of varchar data with 150 MB length is very slow.*

*root casue:  At UnsafeDMStore, ensure memory is just incresing by 8KB each time and lot of time malloc and free happens before reaching 150MB. hence very slow performance.*

*solution: directly check and allocate the required size.*


 *3. Jvm crash when data size is more than 128 MB in unsafe sort step.*

*root cause: unsafeCarbonRowPage is of 128MB, so if data is more than 128MB  for one row, we access block beyond allocated, leading to JVM crash.*

*solution: validate the size before access and prompt user to increase unsafe memory. (by carbon property)*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)