GitHub user kumarvishal09 opened a pull request:
https://github.com/apache/carbondata/pull/2213 [CARBONDATA-2386]Fix Query performance issue with PreAggregate table
**Problem:** Query on pre aggregate table is consuming too much time.
**Root cause:** Time consumption to calculate size of selecting the smallest Pre-Aggregate table is approximately 76 seconds. This is index file is being read when segment file is present to compute the size of Pre-Aggregate table
**Solution:** Read table status and get the size of data file and index file for valid segments. For older segments were datasize and indexsize is not present calculate the size of store folder
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Tested in Cluster with 2 billion rows and 16 aggregate tables
Older Time:76 seconds
New Time: 6 seconds
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/kumarvishal09/incubator-carbondata preagg_perfix
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2213.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2213
----
commit b8857c9e4064f30ec69fd55a6dfb47306e95616e
Author: kumarvishal09 <kumarvishal1802@...>
Date: 2018-04-23T11:33:00Z
Fix performance issue
----
---
Locked
