Apache CarbonData Dev Mailing List archive › Apache CarbonData JIRA issues

[GitHub] carbondata pull request #2212: [CARBONDATA-2313] fixed issue in query when m...

Classic

List

Threaded

7 messages Options

qiuchenjian-2

[GitHub] carbondata pull request #2212: [CARBONDATA-2313] fixed issue in query when m...

GitHub user ajantha-bhat opened a pull request:

https://github.com/apache/carbondata/pull/2212

[CARBONDATA-2313] fixed issue in query when multiple sdk writer's outâ¦

problem: [Non-tranactional table] issue in query when multiple sdk writer's ouput files with same column name with same UUID is placed in single path

scenario: copy one set of sdk writer output, do select query. copy other set of sdk writer output with same column name and same UUID, do select query.
select query result's only first sets output.

root cause : segment map was not updated with the latest files , if it is new files placed belong to same segment.

solution: refresh the segment map, if there is a change in list of carobnindex files during each query for non transactional tables.

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

- [ ] Any interfaces changed?NA

- [ ] Any backward compatibility impacted?NA

- [ ] Document update required?NA

- [ ] Testing done.
done. updated the testcase with same uuid.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ajantha-bhat/carbondata unmanaged_table

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2212.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2212

----
commit 92d85cc706de0345b9a7cfe0eda81f7e92139235
Author: ajantha-bhat <ajanthabhat@...>
Date: 2018-04-23T10:47:29Z

[CARBONDATA-2313] fixed issue in query when multiple sdk writer's output files with same column name with same UUID is placed in single path

----

---

qiuchenjian-2

[GitHub] carbondata pull request #2212: [CARBONDATA-2313] fixed issue in query when m...

Github user gvramana commented on a diff in the pull request:

https://github.com/apache/carbondata/pull/2212#discussion_r183374584

--- Diff: core/src/main/java/org/apache/carbondata/core/indexstore/blockletindex/BlockletDataMapFactory.java ---
@@ -103,6 +105,53 @@ public DataMapWriter createWriter(Segment segment, String writeDirectoryPath) {
indexFile.getName(), indexFileEntry.getValue(), segment.getSegmentNo()));
}
segmentMap.put(segment.getSegmentNo(), tableBlockIndexUniqueIdentifiers);
+ } else {
--- End diff --

This should not be handled using list if files, check at datamap. this logic is common for all data maps. So should be handled using segment update time.

---

qiuchenjian-2