[Discussion] Encryption support for carbondata files

Posted by Ajantha Bhat on Oct 31, 2018; 5:22am
URL: http://apache-carbondata-dev-mailing-list-archive.168.s1.nabble.com/Discussion-Encryption-support-for-carbondata-files-tp67076.html

*Background:* Currently carbondata files are not encrypted. If anyone has
carbon reader, they can read the carbondata files.
If the data has sensitive information, that data can be encrypted with the
crypto key.
So, that along with carbon reader this key is required to decrypt and read
the data.

*Why encryption at file format level ?*
As files generated by one application can be used by the other applications
to read.
Also encrypting the data at application level is a time consuming process
as we have very huge data.
and whole carbondata files need to be encrypted from application. This is
redundant.

Only the columns that have sensitive data can be encrypted if we support
encryption at file format level. so that we can have column level
encryption.

*Note:* Also keep in mind that encryption needs more CPU for crypto key
computation and decryption also takes some time.
So, it will impact loading and query time if user wants to encrypt the data.

*So, how many of you think this feature has real world use case and carbon
should have this feature ?*

Based on the need of this feature, I can go ahead and explore the
implementation details.

Thanks,
Ajantha