[jira] [Created] (CARBONDATA-3448) Wrong results in preaggregate query with spark adaptive execution

[Akash R Nilugal (Jira)]

Ajantha Bhat created CARBONDATA-3448:
----------------------------------------

             Summary: Wrong results in preaggregate query with spark adaptive execution
                 Key: CARBONDATA-3448
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3448
             Project: CarbonData
          Issue Type: Bug
            Reporter: Ajantha Bhat


problem: Wrong results in preaggregate query with spark adaptive execution

Spark2TestQueryExecutor.conf.set(SQLConf.ADAPTIVE_EXECUTION_ENABLED.key, "true")

 

cause: For preaggreagate, segment info is set into threadLocal. when adaptive execution is called, spark is calling getInternalPartition in another thread where updated segment conf  is not set. Hence it is not using the updated segments.

 

solution: CarbonScanRdd is already having the sessionInfo, use it instead of taking session info from the current thread.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)