GitHub user jackylk opened a pull request:
https://github.com/apache/carbondata/pull/2755 [CARBONDATA-2965] Support scan performance benchmark in CarbonCli tool
Scan performance benchmarking is added as a command in CarbonCli tool.
Example usage:
shell>java -jar carbondata-cli.jar org.apache.carbondata.CarbonCli -cmd benchmark -p hdfs://carbon1:9000/carbon.store/tpchcarbon_base/lineitem/ -a -c l_comment
outputs
```
## Benchmark
ReadHeaderAndFooter takes 12,598 us
ConvertFooter takes 4,712 us
ReadAllMetaAndConvertFooter takes 8,039 us
Scan column 'l_comment'
Blocklet#0: ColumnChunkIO takes 222,609 us
Blocklet#0: DecompressPage takes 111,985 us
Blocklet#1: ColumnChunkIO takes 186,522 us
Blocklet#1: DecompressPage takes 89,132 us
Blocklet#2: ColumnChunkIO takes 209,129 us
Blocklet#2: DecompressPage takes 84,051 us
```
- [ ] Any interfaces changed?
- [ ] Any backward compatibility impacted?
- [ ] Document update required?
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
- How it is tested? Please attach test report.
- Is it a performance related change? Please attach the performance test report.
- Any additional information to help reviewers in testing this change.
- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/jackylk/incubator-carbondata scan-test
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2755.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2755
----
commit b8dce3a935c9ea1df45ccc4135834cb2f21c06cf
Author: Jacky Li <jacky.likun@...>
Date: 2018-09-22T16:01:04Z
support Benchmark command in CarbonCli
commit 9fed41fcfb9b1f2900029db5ab0a3a8610b4085e
Author: Jacky Li <jacky.likun@...>
Date: 2018-09-24T15:10:54Z
fix test
----
---
Locked
