Re: Support Zstd as Column Compressor

Posted by xuchuanyin on Sep 13, 2018; 2:22am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Support-Zstd-as-Column-Compressor-tp60417p62364.html

Yeah. Zstd and Snappy knows the size of decompress size from the compressed
data, but LZ4 don't. I find a link to describe this:
https://github.com/lz4/lz4-java/issues/26

To work around with LZ4, you can go with your proposal and save&use the
decompress size in the meta.
But I'd like to wrap the LZ4 implementation by
1. adding original size when we return the compressed content
2. extracting the original size when we want to decompress the content.
In this way, we can make the API stable.




--
Sent from: http://apache-carbondata-dev-mailing-list-archive.1130556.n5.nabble.com/