GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2962 [CARBONDATA-3138] Fix random count mismatch with multi-thread block pruning
problem: Random count mismatch in query in multi-thread block-pruning scenario.
cause: Existing prune method not meant for multi-threading as synchronization was missing.
only in implicit filter scenario, while preparing the block ID list, synchronization was missing. Hence pruning was giving wrong result.
solution: synchronize the implicit filter preparation, as prune now called in multi-thread
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [ ] Any interfaces changed? NA
- [ ] Any backward compatibility impacted? NA
- [ ] Document update required? NA
- [ ] Testing done
done with huge data
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/ajantha-bhat/carbondata issue_fix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2962.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2962
----
commit 09a469e926d43b4992084e5dc1727899ee04366f
Author: ajantha-bhat <ajanthabhat@...>
Date: 2018-11-28T13:48:16Z
random count mismatch with multi-thread block pruning
----
---
Locked
