GitHub user xuchuanyin opened a pull request:
https://github.com/apache/carbondata/pull/2367 [CARBONDATA-2601] Support column with varchar type in DDL as longstring
In DDL statement, column with varchar data type is implicitly considered long string column. User can also explicitly specify the column as string column in property 'long_string_columns'. Carbondata will consider both of them.
This PR follows #2252
Be sure to do all of the following checklist to help us incorporate
your contribution quickly and easily:
- [x] Any interfaces changed?
`NO`
- [x] Any backward compatibility impacted?
`NO`
- [x] Document update required?
`Yes, will update it after the 32k feature is finished`
- [x] Testing done
Please provide details on
- Whether new unit test cases have been added or why no new tests are required?
`Tests update`
- How it is tested? Please attach test report.
`Tested in local`
- Is it a performance related change? Please attach the performance test report.
`NO`
- Any additional information to help reviewers in testing this change.
`NA`
- [x] For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.
`NA`
You can merge this pull request into a Git repository by running:
$ git pull
https://github.com/xuchuanyin/carbondata 0611_support_varchar_ddl_4_longstring
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/carbondata/pull/2367.patchTo close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2367
----
commit a57305aa4a34f905ab719eabd7645f468c3fa6fe
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-02T07:17:04Z
Support string longer than 32000 characters
Add a table property 'long_string_columns' in create table DDL that
indicate those columns will contain more than 32000 characters.
Internally in Carbondata,
1. add a new datatype called `text` to represent the long string column
2. add a new encoding called `DIRECT_COMPRESS_TEXT` to the text column
page meta
3. Use an integer (previously short) to store the length of bytes
content.
commit 558f47d0ba3af5c8c70046aaee4d14e2fd2ea6ac
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-05T12:46:26Z
rename datatype name from text to varchar
commit 0eca63cb1cc6c2298c7a0f16590acf38820947b6
Author: xuchuanyin <xuchuanyin@...>
Date: 2018-06-11T03:15:44Z
Support varchar datatype in DDL as long string
In DDL statement, column with varchar data type is implicitly considered
as long string column. User can also explicitly specify the column as
long string column in property 'long_string_columns'. Carbondata
internally will consider both of them.
----
---
Locked
