Apache CarbonData Dev Mailing List archive › Apache CarbonData JIRA issues

[GitHub] carbondata pull request #2380: [CARBONDATA-2509][CARBONDATA-2510][CARBONDATA...

Classic

List

Threaded

29 messages Options

qiuchenjian-2

[GitHub] carbondata pull request #2380: [CARBONDATA-2509][CARBONDATA-2510][CARBONDATA...

GitHub user xuchuanyin opened a pull request:

https://github.com/apache/carbondata/pull/2380

[CARBONDATA-2509][CARBONDATA-2510][CARBONDATA-2511][32K] Add validate for long string columns

1. long string columns cannot be sort_columns
2. long string columns cannot be dictionary include
3. long string columns cannot be dictionary exclude
4. long string columns can only be string columns

Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:

- [x] Any interfaces changed?
`NO`
- [ ] Any backward compatibility impacted?
`NO`
- [ ] Document update required?
`NO`
- [ ] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
`Added tests`
- How it is tested? Please attach test report.
`Tested in local`
- Is it a performance related change? Please attach the performance test report.
`NO`
- Any additional information to help reviewers in testing this change.

- [ ] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/xuchuanyin/carbondata 0620_longstring_nodict_nosort

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/carbondata/pull/2380.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2380

----
commit b689d66493521452ff9938415e0d0aa66b56c2c5
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-02T07:17:04Z

Support string longer than 32000 characters

Add a table property 'long_string_columns' in create table DDL that
indicate those columns will contain more than 32000 characters.

Internally in Carbondata,
1. add a new datatype called `text` to represent the long string column
2. add a new encoding called `DIRECT_COMPRESS_TEXT` to the text column
page meta
3. Use an integer (previously short) to store the length of bytes
content.

commit f145c6c60238c400b5db6a6bf2696246b698154a
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-05T12:46:26Z

rename datatype name from text to varchar

commit 4180f8118d1ff90205b0f1567bef2cdfee3a1b62
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-12T12:35:58Z

Add 2GB constraint for one column page

commit 710845b155ed5b7a611a900c70b0d766d80ae48d
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-14T12:11:40Z

update tests

commit b994cee490b82d8ec0a1a4cefec6a3523785b016
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-19T07:11:21Z

Add validate for long string columns

1. long string columns cannot be sort_columns
2. long string columns cannot be dictionary include
3. long string columns cannot be dictionary exclude
4. long string columns can only be string columns

----

---

qiuchenjian-2

[GitHub] carbondata issue #2380: [CARBONDATA-2509][CARBONDATA-2510][CARBONDATA-2511][...

Github user xuchuanyin commented on the issue:

https://github.com/apache/carbondata/pull/2380

This PR depends on #2379

---

qiuchenjian-2