Recently some of us encount a problem in loading when the executor side
throw an OOM exception but the driver side said "unable to generate the
mdkey". Actually no one will throw KeyGenException. Anyway, the error
message in driver is unrelevant.
Dive into the code, we found that it is caused by Wrapped Exception as
below. Actual exception is throw in addDataToStore when new a column page.
We should make the error message more helpful for debugging instead of
becoming confusing.
```
private void processBatch(CarbonRowBatch batch, CarbonFactHandler
dataHandler, int iteratorIndex)
throws CarbonDataLoadingException {
try {
while (batch.hasNext()) {
CarbonRow row = batch.next();
CarbonRow converted = convertRow(row);
dataHandler.addDataToStore(converted);
readCounter[iteratorIndex]++;
}
writeCounter[iteratorIndex] += batch.getSize();
} catch (Exception e) {
throw new CarbonDataLoadingException("unable to generate the mdkey",
e);
}
rowCounter.getAndAdd(batch.getSize());
}
```
By using regex expression, we found so many codes of "catch \([^}]*throw
new" in java and "catch \{[^}]*throw new" in scala to deal with exception.
Any good ideas to decide whether there is problem for the wrapped exception
and how to fix it?
For this example, shall we change the error message to "e.getMessage()" to
keep original error?
-----
Regards
Manhua
--
Sent from:
http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/
Regards
Manhua