[GitHub] carbondata pull request #3028: [CARBONDATA-3205]Fix Get Local Dictionary for...

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #3028: [CARBONDATA-3205]Fix Get Local Dictionary for...

qiuchenjian-2
GitHub user Indhumathi27 opened a pull request:

    https://github.com/apache/carbondata/pull/3028

    [CARBONDATA-3205]Fix Get Local Dictionary for empty Array of Struct

    Problem:
    When a table contains a nested complex schema  with Array of Struct as empty data, get dictionary for empty Array of Struct fails.
   
    Solution:
    Check if usedDictionary is empty, then return null
   
     - [ ] Any interfaces changed?
     
     - [ ] Any backward compatibility impacted?
     
     - [ ] Document update required?
   
     - [x] Testing done
          Added a test case
           
     - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
   


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/Indhumathi27/carbondata nested

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/carbondata/pull/3028.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3028
   
----
commit 99e6233dcdb4b488d0f0366ac5966d4e9c4f895c
Author: Indhumathi27 <indhumathim27@...>
Date:   2018-12-27T12:02:48Z

    [CARBONDATA-3205]Fix Get Local Dictionary for empty Array of Struct

----


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #3028: [CARBONDATA-3205]Fix Get Local Dictionary for...

qiuchenjian-2
Github user qiuchenjian commented on a diff in the pull request:

    https://github.com/apache/carbondata/pull/3028#discussion_r244142318
 
    --- Diff: core/src/main/java/org/apache/carbondata/core/datastore/chunk/impl/DimensionRawColumnChunk.java ---
    @@ -197,6 +197,10 @@ public static CarbonDictionary getDictionary(LocalDictionaryChunk localDictionar
               localDictionaryChunk.getDictionary_data().length);
           BitSet usedDictionary = BitSet.valueOf(compressor.unCompressByte(
               localDictionaryChunk.getDictionary_values()));
    +      // if dictionary is empty, return null
    +      if (usedDictionary.isEmpty()) {
    --- End diff --
   
    DataFile class call this method,but it use the return value directly, it will throw NullException. i think it need be protected
    Please see DataFile line 360,
    CarbonDictionary dictionary = DimensionRawColumnChunk.getDictionary(dictionaryChunk, comp);
    blockletDictionaryEntries = dictionary.getDictionaryActualSize();
   



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #3028: [CARBONDATA-3205]Fix Get Local Dictionary for empty ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3028
 
    Build Success with Spark 2.2.1, Please check CI http://95.216.28.178:8080/job/ApacheCarbonPRBuilder1/2255/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #3028: [CARBONDATA-3205]Fix Get Local Dictionary for empty ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3028
 
    Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder2.1/2048/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #3028: [CARBONDATA-3205]Fix Get Local Dictionary for empty ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:

    https://github.com/apache/carbondata/pull/3028
 
    Build Success with Spark 2.3.2, Please check CI http://136.243.101.176:8080/job/carbondataprbuilder2.3/10300/



---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata issue #3028: [CARBONDATA-3205]Fix Get Local Dictionary for empty ...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:

    https://github.com/apache/carbondata/pull/3028
 
    LGTM


---
Reply | Threaded
Open this post in threaded view
|

[GitHub] carbondata pull request #3028: [CARBONDATA-3205]Fix Get Local Dictionary for...

qiuchenjian-2
In reply to this post by qiuchenjian-2
Github user asfgit closed the pull request at:

    https://github.com/apache/carbondata/pull/3028


---