[jira] [Resolved] (CARBONDATA-3426) Fix Load performance degrade by fixing task distribution

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

[jira] [Resolved] (CARBONDATA-3426) Fix Load performance degrade by fixing task distribution

Akash R Nilugal (Jira)

     [ https://issues.apache.org/jira/browse/CARBONDATA-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Kunal Kapoor resolved CARBONDATA-3426.
--------------------------------------
       Resolution: Fixed
    Fix Version/s: 1.6.0

> Fix Load performance degrade by fixing task distribution
> --------------------------------------------------------
>
>                 Key: CARBONDATA-3426
>                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3426
>             Project: CarbonData
>          Issue Type: Bug
>            Reporter: Ajantha Bhat
>            Priority: Major
>             Fix For: 1.6.0
>
>          Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Problem: Load performance degrade by fixing task distribution issue.
> Cause: Consider 3 node cluster (host name a,b,c with IP1, IP2, IP3 as ip address), to launch load task, host name is required from NewCarbonDataLoadRDD in getPreferredLocations(). But if the driver is a (IP1), 
> result is IP1, b,c instead of a,b,c. Hence task was not launching to one executor which is same ip as driver.
> getLocalhostIPs is modified in current version recently and instead of IP it was returning address, hence local ip hostanme was removed instead of address.
> solution: Revert the change in getLocalhostIPs as it is not used in any other flow.  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)