GitHub user ravipesala opened a pull request:
https://github.com/apache/incubator-carbondata/pull/576 [WIP] Improve query performance with read a head blocklet and changed number compressor. Need to work out with backward compatability You can merge this pull request into a Git repository by running: $ git pull https://github.com/ravipesala/incubator-carbondata vector_perform Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/576.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #576 ---- commit 1c64874f18610be5de1490fe4f450ea6ca2127b0 Author: ravipesala <[hidden email]> Date: 2017-01-12T06:37:41Z Added prefetch to scanner commit d61ef31ff3867bd0a1e032e4753a99a9db54e519 Author: ravipesala <[hidden email]> Date: 2017-01-26T09:54:24Z Added read a head blocklet commit 37d7ca08c7184d01eba2eb717f9c204ed8817142 Author: ravipesala <[hidden email]> Date: 2017-01-27T07:13:22Z Added read a head commit 6b3af88bf8f19c3156d9c3e5c8208bc13cf10018 Author: ravipesala <[hidden email]> Date: 2017-01-27T07:19:12Z Fixed style ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/757/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/759/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/761/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user jackylk commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/576#discussion_r98163965 --- Diff: core/src/main/java/org/apache/carbondata/core/keygenerator/mdkey/NumberCompressor.java --- @@ -137,12 +161,31 @@ public NumberCompressor(int cardinality) { return words; } + // public int[] unCompress(byte[] key, int offset, int length) { + // int ls = length; + // int arrayLength = (ls * BYTE_LENGTH) / bitsLength; + // long[] words = new long[getWordsSizeFromBytesSize(ls)]; + // unCompressVal(key, ls, words, offset); + // return getArray(words, arrayLength); + // } + public int[] unCompress(byte[] key, int offset, int length) { - int ls = length; - int arrayLength = (ls * BYTE_LENGTH) / bitsLength; - long[] words = new long[getWordsSizeFromBytesSize(ls)]; - unCompressVal(key, ls, words, offset); - return getArray(words, arrayLength); + int len = length / bytesPerKey; + if (length % bytesPerKey != 0) { + throw new IllegalArgumentException( + "Some thing wrong while decompress" + length + " " + bytesPerKey); --- End diff -- Both length is read from footer, right? If comes here, mean the file is corrupted. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/762/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/790/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Build Failed with Spark 1.6.2, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder/801/ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/incubator-carbondata/pull/576 Same has been handled in https://github.com/apache/incubator-carbondata/pull/584, so closing this PR. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
In reply to this post by qiuchenjian-2
Github user ravipesala closed the pull request at:
https://github.com/apache/incubator-carbondata/pull/576 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at [hidden email] or file a JIRA ticket with INFRA. --- |
Free forum by Nabble | Edit this page |