[GitHub] carbondata pull request #3062: [WIP][CARBONDATA-3241] Refactor the requested...

[qiuchenjian-2]

GitHub user dhatchayani opened a pull request:

    https://github.com/apache/carbondata/pull/3062

    [WIP][CARBONDATA-3241] Refactor the requested scan columns and the projection columns

    Refactor the requested columns methods by changing both the scan list and the projection list together.
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [x] Testing done
            Existing UT
   
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dhatchayani/carbondata CARBONDATA-3241

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/3062.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3062
   
----
commit 5f106d70737bd3982df855144f8d7bc18635c634
Author: dhatchayani <dhatcha.official@...>
Date:   2019-01-10T09:30:51Z

    [CARBONDATA-3241] Refactor the requested scan columns and the projection columns

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2250/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2469/



---

[qiuchenjian-2]

Github user qiuchenjian commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/3062#discussion_r246715623
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/strategy/CarbonLateDecodeStrategy.scala ---
    @@ -449,9 +455,10 @@ private[sql] class CarbonLateDecodeStrategy extends SparkStrategy {
           projectsAttr: Seq[Attribute],
           filterSet: AttributeSet,
           handledSet: AttributeSet,
    -      newProjectList: Seq[Attribute]) = {
    -    (projectsAttr.to[mutable.LinkedHashSet] ++ filterSet -- handledSet)
    -      .map(relation.attributeMap).toSeq ++ newProjectList
    +      newProjectList: Seq[Attribute],
    +      updatedProjects: Seq[Expression]): (Seq[Attribute], Seq[Expression]) = {
    --- End diff --
   
    it seems that new param of updatedProjects do nothing in getRequestedColumns


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Failed  with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10507/



---

[qiuchenjian-2]

Github user dhatchayani commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2251/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2470/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3062
 
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10508/



---