[jira] [Resolved] (CARBONDATA-3761) Remove redundant conversion for complex type insert

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-3761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacky Li resolved CARBONDATA-3761.
----------------------------------
    Fix Version/s: 2.0.0
       Resolution: Fixed

> Remove redundant conversion for complex type insert
> ---------------------------------------------------
>
>                 Key: CARBONDATA-3761
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3761
>             Project: CarbonData
>          Issue Type: Improvement
>            Reporter: Ajantha Bhat
>            Assignee: Ajantha Bhat
>            Priority: Major
>             Fix For: 2.0.0
>
>          Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> h3. Problem
>  # In {{PrimitiveDataType#writeByteArray}}
> {{DataTypeUtil.parseValue(**input.toString()**, carbonDimension)}}
> Here we convert every complex child element to string and then parse as an object to handle bad records. Which leads to heavy GC
>  # {{DatatypeUtil#getBytesDataDataTypeForNoDictionaryColumn}} -> double,float, byte, decimal case is missing. so we convert them to string and then convert to bytes. which create more redundant objects
> h3. Solution
>  # For new Insert into flow, no need to handle bad records for complex types as it is already validated in source table. So, use object directly. This can decrease the memory foot print for complex type insert
>  # Add a case for double,float, byte, decimal data type.

--
This message was sent by Atlassian Jira
(v8.3.4#803005)