[GitHub] carbondata pull request #1510: [WIP] Supported DataMap chooser and expressio...

[qiuchenjian-2]

GitHub user ravipesala opened a pull request:

    https://github.com/apache/carbondata/pull/1510

    [WIP] Supported DataMap chooser and expression for supporting multiple datamaps in single query

      This PR supports 3 features.
     1. Load datamaps from the DataMapSchema which are created through DDL.
     2. DataMap Chooser: It chooses the datamap out of available datamaps based on simple logic. Like if there is filter condition on column1 then for supposing 2 datamaps(1. column1 2. column1+column2) are supporting this column then we choose the datamap which has fewer columns that is the first datamap.
     3. Expression support: Based on the filter expressions we convert them to the possible DataMap expressions and do apply expression on it.
      For example, there are 2 datamaps available on table1
      Datamap1 : column1
      Datamap2 : column2
      Query: `select * from table1 where column1 ='a' and column2 = `b` `
      For the above query, we create datamap expression as `AndDataMapExpression(Datamap1, DataMap2)`. So for the above query both the datamaps are included and the output of them will be applied AND condition to improve the performance
   
     - [X] Any interfaces changed?
       Yes, DataMap interfaces are changed in order to support datamap expression.
     - [X] Any backward compatibility impacted?
       NO
     
     - [X] Document update required?
       Yes, dev guide need to be updated
     - [X] Testing done
           Tests are added
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/ravipesala/incubator-carbondata datamap-fg-chooser1

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1510.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1510
   
----
commit c2b898468acdfb3d1738c34ce0557100700f6f80
Author: ravipesala <[hidden email]>
Date:   2017-11-15T14:18:40Z

    Added FG implementation and testcases

commit 3a405355874c051fd640cc3ad00049a1a73fd8c6
Author: ravipesala <[hidden email]>
Date:   2017-11-08T16:00:32Z

    Added DataMapChooser

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1185/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1277/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1331/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1790/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1332/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1349/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1804/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1350/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1805/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1806/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1352/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1807/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1355/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1859/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1435/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1449/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1868/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1450/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1510
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1869/



---