[jira] [Resolved] (CARBONDATA-2702) Fix bugs in clear bloom datamap

[Akash R Nilugal (Jira)]

     [ https://issues.apache.org/jira/browse/CARBONDATA-2702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Jacky Li resolved CARBONDATA-2702.
----------------------------------
       Resolution: Fixed
    Fix Version/s: 1.4.1
                   1.5.0

> Fix bugs in clear bloom datamap
> -------------------------------
>
>                 Key: CARBONDATA-2702
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2702
>             Project: CarbonData
>          Issue Type: Sub-task
>            Reporter: xuchuanyin
>            Assignee: xuchuanyin
>            Priority: Major
>             Fix For: 1.5.0, 1.4.1
>
>          Time Spent: 2.5h
>  Remaining Estimate: 0h
>
> In test output, sometimes there are error logs like:
> ```
> org.apache.spark.SparkException: Job aborted due to stage failure: Task 3 in stage 805.0 failed 1 times, most recent failure: Lost task 3.0 in stage 805.0 (TID 5751, localhost, executor driver): java.lang.NegativeArraySizeException
> at java.util.AbstractCollection.toArray(AbstractCollection.java:136)
> at java.util.ArrayList.<init>(ArrayList.java:177)
> at org.apache.carbondata.datamap.bloom.BloomCoarseGrainDataMapFactory.clear(BloomCoarseGrainDataMapFactory.java:340)
> at org.apache.carbondata.core.datamap.TableDataMap.clear(TableDataMap.java:206)
> at org.apache.carbondata.core.datamap.DataMapStoreManager.clearDataMaps(DataMapStoreManager.java:430)
> at org.apache.carbondata.core.datamap.DistributableDataMapFormat$1.initialize(DistributableDataMapFormat.java:125)
> at org.apache.carbondata.spark.rdd.DataMapPruneRDD.internalCompute(SparkDataMapJob.scala:73)
> ```
> This error is caused by concurrent clear for datamaps, can refer to BlockletDataMapFactory at issue 2496 (PR2324)

--
This message was sent by Atlassian JIRA
(v7.6.3#76005)