[
https://issues.apache.org/jira/browse/CARBONDATA-3695?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Ajantha Bhat resolved CARBONDATA-3695.
--------------------------------------
Fix Version/s: 2.0.0
Resolution: Fixed
> Apache CarbonData should provides python interface to support deep learning framework PyTorch to ready and write data from/to CarbonData
> ----------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CARBONDATA-3695
> URL:
https://issues.apache.org/jira/browse/CARBONDATA-3695> Project: CarbonData
> Issue Type: Sub-task
> Reporter: Bo Xu
> Assignee: Bo Xu
> Priority: Major
> Fix For: 2.0.0
>
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> Nowadays AI model training is getting more and more popular. Currently many AI framework uses raw data files or row format data files for model training, it could not provide projection, filtering, and fast scan capability like in columnar store. So, if CarbonData supports AI framework, it can speed up model training by increase IO throughput, and provide more flexible training set selection ability to AI developers
> AI compute engine integration:
> PyTorch integration: New python API in pycarbon to support PyTorch to read data from CarbonData files for training model
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Locked
