[GitHub] carbondata pull request #1542: [CARBONDATA-1757] [PreAgg] Fix for wrong avg ...

[qiuchenjian-2]

GitHub user kunal642 opened a pull request:

    https://github.com/apache/carbondata/pull/1542

    [CARBONDATA-1757] [PreAgg] Fix for wrong avg values after pre-agg table creation

    when a sum/count aggregation function is applied on the same column along with avg. The plan that was getting transformed was adding 2 columns for sum/count which resulted in wrong data being inserted.
   
    Be sure to do all of the following checklist to help us incorporate
    your contribution quickly and easily:
   
     - [X] Any interfaces changed?
     No
     - [X] Any backward compatibility impacted?
     No
     - [X] Document update required?
    No
     - [X] Testing done
    Test case added
     - [X] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
    No


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kunal642/carbondata pre_agg_avg_fix

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/1542.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1542
   
----
commit f05ba4faabc70c19574ed200ddb88909d69cf6e3
Author: kunal642 <[hidden email]>
Date:   2017-11-21T07:25:25Z

    fixed wrong avg count bug when a sum/count aggregation function is applied on the same column along with avg.
    The plan that was getting transformed was adding 2 columns for sum/count which resulted in wrong data being inserted.

----


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1329/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1330/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1788/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1789/



---

[qiuchenjian-2]

Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1334/



---

[qiuchenjian-2]

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1542#discussion_r152357935
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateUtil.scala ---
    @@ -489,4 +489,44 @@ object PreAggregateUtil {
         }
         updatedPlan
       }
    +
    --- End diff --
   
    This code is specific for inserting the data to aggregate table better to move this code in rules class


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1353/



---

[qiuchenjian-2]

Github user kunal642 commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/1542#discussion_r152474076
 
    --- Diff: integration/spark2/src/main/scala/org/apache/spark/sql/execution/command/preaaggregate/PreAggregateUtil.scala ---
    @@ -489,4 +489,44 @@ object PreAggregateUtil {
         }
         updatedPlan
       }
    +
    --- End diff --
   
    moved the method to PreAggregationRules


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1354/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1808/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1809/



---

[qiuchenjian-2]

Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1373/



---

[qiuchenjian-2]

Github user kunal642 commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    retest this please


---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1385/



---

[qiuchenjian-2]

Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    Build Failed  with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/1387/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Fail , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1834/



---

[qiuchenjian-2]

Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/1542
 
    SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/1836/



---