[jira] [Created] (CARBONDATA-3056) Implement Concurrent SDK Reader

[Akash R Nilugal (Jira)]

Naman Rastogi created CARBONDATA-3056:
-----------------------------------------

             Summary: Implement Concurrent SDK Reader
                 Key: CARBONDATA-3056
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3056
             Project: CarbonData
          Issue Type: Sub-task
            Reporter: Naman Rastogi


The current reading through SDK is slow as in CarbonReader, we are reading the carbondata files sequentially, even though we have individual CarbonRecordReader for each file. We can parallelize this by adding an API in CarbonReader class
*List<CarbonReader> readers = CarbonReader.split(numSplits)*
which returns a list of CarbonReaders, which can be used to read parallelly, as reading each file is independent of other files.
 
This enables the SDK user to read the files as it is, or in a multithreaded environment.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)