[jira] [Updated] (CARBONDATA-2814) Refactor FileFactory and related code to take hadoop configuration from spark session

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Updated] (CARBONDATA-2814) Refactor FileFactory and related code to take hadoop configuration from spark session

Akash R Nilugal (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-2814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor updated CARBONDATA-2814:
-------------------------------------
    Summary: Refactor FileFactory and related code to take hadoop configuration from spark session  (was: Refactor FileFactory and related code to take hadoop configuration from spark sessoion)

> Refactor FileFactory and related code to take hadoop configuration from spark session
> -------------------------------------------------------------------------------------
>
>                 Key: CARBONDATA-2814
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-2814
>             Project: CarbonData
>          Issue Type: Sub-task
>            Reporter: Kunal Kapoor
>            Assignee: Kunal Kapoor
>            Priority: Critical
>
> Currently File Factory in core module creates HadoopConf using core-site.xml and as this is done in a static block therefore it is unable to get any dynamically set properties like SK and AK when set using spark set command. 
>  
> The proposed solution is to refactor the code of FileFactory so that it can accept configuration from the caller and the same configuration should be passed to the executors from driver.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)