[jira] [Updated] (CARBONDATA-247) Higher MAXCOLUMNS value in load DML options is leading to out of memory error

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Venkata Ramana G updated CARBONDATA-247:
----------------------------------------
    Component/s: data-load

> Higher MAXCOLUMNS value in load DML options is leading to out of memory error
> -----------------------------------------------------------------------------
>
>                 Key: CARBONDATA-247
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-247
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>            Reporter: dhatchayani
>            Priority: Minor
>             Fix For: 0.2.0-incubating
>
>
> When a higher value lets say Integer max value is configured for maxcolumns option in load DML and executor memory is less, then in that case UnivocityCsvParser throws an out of memory error when it tries to create an array of size of maxColumns option value.
> java.lang.OutOfMemoryError: Java heap space
> at com.univocity.parsers.common.ParserOutput.<init>(ParserOutput.java:86)
> at com.univocity.parsers.common.AbstractParser.<init>(AbstractParser.java:66)
> at com.univocity.parsers.csv.CsvParser.<init>(CsvParser.java:50)
> at org.apache.carbondata.processing.csvreaderstep.UnivocityCsvParser.initialize(UnivocityCsvParser.java:114)
> at org.apache.carbondata.processing.csvreaderstep.CsvInput.doProcessUnivocity(CsvInput.java:427)
> at org.apache.carbondata.processing.csvreaderstep.CsvInput.access$100(CsvInput.java:60)
> at org.apache.carbondata.processing.csvreaderstep.CsvInput$1.call(CsvInput.java:389)

... [show rest of quote]

--
This message was sent by Atlassian JIRA
(v6.3.4#6332)