[jira] [Commented] (CARBONDATA-658) Compression is not working for BigInt and Int datatype

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Commented] (CARBONDATA-658) Compression is not working for BigInt and Int datatype

Akash R Nilugal (Jira)

    [ https://issues.apache.org/jira/browse/CARBONDATA-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15842315#comment-15842315 ]

Ravindra Pesala commented on CARBONDATA-658:
--------------------------------------------

Yes, it uses delta compression to compress it. So there is no issue,Please close this jira.

> Compression is not working for BigInt and Int datatype
> ------------------------------------------------------
>
>                 Key: CARBONDATA-658
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-658
>             Project: CarbonData
>          Issue Type: Bug
>          Components: data-load
>    Affects Versions: 1.0.0-incubating
>         Environment: spark 1.6, 2.0
>            Reporter: Geetika Gupta
>         Attachments: 100000_LargeBigInt.csv, 100000_LargeInt.csv, 100000_SmallBigInt.csv, 100000_SmallInt.csv, sample1.csv
>
>
> I tried to load data into a table having bigInt as a column. Firstly I loaded small bigint values to the table and noted down the carbondata file size then I loaded max bigint values to the table and again noted the carbondata file size.
> For large bigint values the carbondata file size was 684.25 Kb and for small bigint values it was 684.26 Kb. So I could not figure out whether compression is performed or not.
> I tried the same scenario with int datatype as well. For large int values the carbondata file size was 684.24 Kb and for small int values it was 684.26 Kb.
> Below are the queries:
> For BigInt table:
> Create table test(a BigInt, b String) stored by 'carbondata';
> LOAD DATA INPATH 'hdfs://localhost:54311/testFiles/100000_LargeBigInt.csv' into table test OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','FILEHEADER'='b,a');
> LOAD DATA INPATH 'hdfs://localhost:54311/testFiles/100000_SmallBigInt.csv' into table test OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','FILEHEADER'='b,a');
> For Int table:
> Create table test(a Int, b String) stored by 'carbondata';
> LOAD DATA INPATH 'hdfs://localhost:54311/testFiles/100000_LargeInt.csv' into table test OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','FILEHEADER'='b,a');
> LOAD DATA INPATH 'hdfs://localhost:54311/testFiles/100000_SmallInt.csv' into table test OPTIONS('DELIMITER'=',' , 'QUOTECHAR'='"','FILEHEADER'='b,a');



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)