[jira] [Commented] (CARBONDATA-224) Fixed data mismatch issue in case of Dictionary Exclude column for Numeric data type

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CARBONDATA-224) Fixed data mismatch issue in case of Dictionary Exclude column for Numeric data type

Akash R Nilugal (Jira)

    [ https://issues.apache.org/jira/browse/CARBONDATA-224?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15472934#comment-15472934 ]

ASF GitHub Bot commented on CARBONDATA-224:
-------------------------------------------

Github user kumarvishal09 commented on a diff in the pull request:

    https://github.com/apache/incubator-carbondata/pull/139#discussion_r77950911
 
    --- Diff: integration/spark/src/main/scala/org/apache/spark/sql/CarbonSqlParser.scala ---
    @@ -864,8 +864,8 @@ class CarbonSqlParser()
        /**
         * detects whether double or decimal column is part of dictionary_exclude
         */
    -  def isDoubleDecimalColDictionaryExclude(columnDataType: String): Boolean = {
    -    val dataTypes = Array("double", "decimal")
    +  def isNumberInColDictionaryExclude(columnDataType: String): Boolean = {
    +     val dataTypes = Array("double", "decimal", "int")
    --- End diff --
   
    ok


> Fixed data mismatch issue in case of Dictionary Exclude column for Numeric data type
> ------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-224
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-224
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: kumar vishal
>            Assignee: kumar vishal
>
> Problem: In case of greater than query on dictionary exclude column of for numeric data type
> This is because data is sorted based on string because of this if data is 1,10,2,3 , data will be sorted like 1,10,2,3 but if we search greater than 3 then while applying min max will return false as last value is 3
> Solution:we need to sort based on actual data type for this we should have chain comparator based on data type while loading the data, currently disabling DictionaryExclude column for numeric data type and will throw exception. Will raise jira issue to for sorting the based on actual data type



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)