[GitHub] carbondata pull request #1674: [CARBONDATA-1859][CARBONDATA-1861] Support sh...

[qiuchenjian-2]

GitHub user ravipesala opened a pull request:

    https://github.com/apache/carbondata/pull/1674

    [CARBONDATA-1859][CARBONDATA-1861] Support show and drop partitions

    This PR depends on https://github.com/apache/carbondata/pull/1642 and https://github.com/apache/carbondata/pull/1654 and https://github.com/apache/carbondata/pull/1672
   
    It supports show and drop partitions.
   
    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [X] Any interfaces changed?
     - [X] Any backward compatibility impacted?
       NO
     - [X] Document update required?
       Yes
     - [X] Testing done
            Tests added
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ravipesala/incubator-carbondata drop-partition

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1674.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1674
   
----
commit 6b49e6c4d9e5a960a657097487e94a83012fe491
Author: ravipesala <[hidden email]>
Date:   2017-12-04T10:37:03Z

    Added outputformat for carbon

commit d8a88314b0a19b8e30a3b1ad1a3540f7f0964c7c
Author: ravipesala <[hidden email]>
Date:   2017-12-12T06:12:45Z

    Added fileformat in carbon

commit 67c4f83c5b62f03dffb2fe8a8aa53923a889d411
Author: ravipesala <[hidden email]>
Date:   2017-12-15T19:18:19Z

    Added support to query using standard partitions

commit a127ece350c1a9b1a1c7755cb51e4e1f9342cac3
Author: ravipesala <[hidden email]>
Date:   2017-12-16T17:08:00Z

    Added drop partition feature

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2056/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/829/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2365/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/833/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2060/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2369/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/840/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2066/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2371/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2397/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2109/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/885/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/2398/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed with Spark 2.2.0, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/886/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1674
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/2111/



---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1674#discussion_r157675550
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/DataMap.java ---
    @@ -42,6 +42,17 @@
        */
       List<Blocklet> prune(FilterResolverIntf filterExp);
     
    +  // TODO Move this method to Abstract class
    --- End diff --
   
    This TODO is not valid now, right?


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1674#discussion_r157675599
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/DataMap.java ---
    @@ -42,6 +42,17 @@
        */
       List<Blocklet> prune(FilterResolverIntf filterExp);
     
    +  // TODO Move this method to Abstract class
    +  /**
    +   * Prune the datamap with filter expression and partition information. It returns the list of
    +   * blocklets where these filters can exist.
    +   *
    +   * @param filterExp
    +   * @return
    +   */
    +  List<Blocklet> prune(FilterResolverIntf filterExp, List<String> partitions);
    +
    +  // TODO Move this method to Abstract class
    --- End diff --
   
    This TODO is not valid now, right?


---

[qiuchenjian-2]

Github user jackylk commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1674#discussion_r157675696
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/BlockletDataMapIndexStore.java ---
    @@ -71,12 +72,14 @@ public BlockletDataMap get(TableBlockIndexUniqueIdentifier identifier)
         BlockletDataMap dataMap = (BlockletDataMap) lruCache.get(lruCacheKey);
         if (dataMap == null) {
           try {
    +        String segmentPath = CarbonTablePath
    --- End diff --
   
    change like:
    ```
    String segmentPath = CarbonTablePath.getSegmentPath(
         identifier.getAbsoluteTableIdentifier().getTablePath(),
         identifier.getSegmentId());
    ```


---

[qiuchenjian-2]

Github user ravipesala commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1674#discussion_r157675898
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datamap/dev/DataMap.java ---
    @@ -42,6 +42,17 @@
        */
       List<Blocklet> prune(FilterResolverIntf filterExp);
     
    +  // TODO Move this method to Abstract class
    +  /**
    +   * Prune the datamap with filter expression and partition information. It returns the list of
    +   * blocklets where these filters can exist.
    +   *
    +   * @param filterExp
    +   * @return
    +   */
    +  List<Blocklet> prune(FilterResolverIntf filterExp, List<String> partitions);
    +
    +  // TODO Move this method to Abstract class
    --- End diff --
   
    In FG implementation I already created Abstract classes , so when we merge it we can move there


---