GitHub user ajantha-bhat opened a pull request:
https://github.com/apache/carbondata/pull/2485 [CARBONDATA-2722] [CARBONDATA-2721] JsonWriter issue fixes [CARBONDATA-2722][SDK] [JsonWriter] NPE when schema and data are not of same length or Data is null. problem: Null data is not handled in the json object to carbon row conversion. solution: add a null check when object is fetched from json map. [CARBONDATA-2721][SDK] [JsonWriter] Json writer is writing only first element of an array and discarding the rest of the elements. problem: converting json object to carbon row array object is based on array children count , instead of array element count. Hence as array will always have one children. only one element is filled. solution: use array element count instead of array children count.Be sure to do all of the following checklist to help us incorporate your contribution quickly and easily: - [ ] Any interfaces changed?NA - [ ] Any backward compatibility impacted?NA - [ ] Document update required?NA - [ ] Testing done updated the UT - [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ajantha-bhat/carbondata master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/carbondata/pull/2485.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2485 ---- commit 1152176762cba7c6b4686fbc2d7825c82e373e3c Author: ajantha-bhat <ajanthabhat@...> Date: 2018-07-09T13:15:30Z [CARBONDATA-2722][SDK] [JsonWriter] NPE when schema and data are not of same length or Data is null. problem: Null data is not handled in the json object to carbon row conversion. solution: add a null check when object is fetched from json map. [CARBONDATA-2721][SDK] [JsonWriter] Json writer is writing only first element of an array and discarding the rest of the elements. problem: converting json object to carbon row array object is based on array children count , instead of array element count. Hence as array will always have one children. only one element is filled. solution: use array element count instead of array children count. ---- --- |
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5791/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7015/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5767/ --- |
In reply to this post by qiuchenjian-2
Github user ajantha-bhat commented on the issue:
https://github.com/apache/carbondata/pull/2485 retest this please --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7051/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Failed with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5828/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Success with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7066/ --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5843/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5809/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2485#discussion_r202256075 --- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/JsonRowParser.java --- @@ -56,6 +56,9 @@ public JsonRowParser(DataField[] dataFields) { Map<String, Object> jsonNodeMap = objectMapper.readValue(jsonString, new TypeReference<Map<String, Object>>() { }); + if (jsonNodeMap == null) { --- End diff -- Here `|| jsonNodeMap.size() == 0` check is not required? --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Failed with Spark 2.1.0, Please check CI http://136.243.101.176:8080/job/ApacheCarbonPRBuilder1/7087/ --- |
In reply to this post by qiuchenjian-2
Github user ajantha-bhat commented on a diff in the pull request:
https://github.com/apache/carbondata/pull/2485#discussion_r202277326 --- Diff: processing/src/main/java/org/apache/carbondata/processing/loading/parser/impl/JsonRowParser.java --- @@ -56,6 +56,9 @@ public JsonRowParser(DataField[] dataFields) { Map<String, Object> jsonNodeMap = objectMapper.readValue(jsonString, new TypeReference<Map<String, Object>>() { }); + if (jsonNodeMap == null) { --- End diff -- yeah, not required. removed it. --- |
In reply to this post by qiuchenjian-2
Github user CarbonDataQA commented on the issue:
https://github.com/apache/carbondata/pull/2485 Build Success with Spark 2.2.1, Please check CI http://88.99.58.216:8080/job/ApacheCarbonPRBuilder/5864/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485 SDV Build Success , Please check CI http://144.76.159.231:8080/job/ApacheSDVTests/5823/ --- |
In reply to this post by qiuchenjian-2
Github user ravipesala commented on the issue:
https://github.com/apache/carbondata/pull/2485 LGTM --- |
In reply to this post by qiuchenjian-2
|
Free forum by Nabble | Edit this page |