GitHub user kumarvishal09 opened a pull request:
https://github.com/apache/carbondata/pull/1384 [CARBONDATA-1515]Fixed NPE in data loading in long run
**Scenario:**
Data size: 3.5 billion rows(4.1 tb data)
3 node cluster
Number of core while data loading 12.
No. of loads 100 times
**Problem:** In DataConverterProcessorStepImpl it is using array list for adding all the local converter, in case of multiple thread scenario it is creating a hole (null value)(as array list if not synchronized). while closing the converter it is it is throwing NPE
**Solution:** Add local converter in synchronized block
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/kumarvishal09/incubator-carbondata NPEInDataLoading
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/1384.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1384
----
commit 7991ca02205fe37d63d6bfa1307a4ca48dd057e1
Author: kumarvishal <
[hidden email]>
Date: 2017-09-25T12:57:03Z
Fixed NPE in data loading
----
---
Locked
