[jira] [Created] (CARBONDATA-3026) clear expired property that may cause GC problem

[Akash R Nilugal (Jira)]

xuchuanyin created CARBONDATA-3026:
--------------------------------------

             Summary: clear expired property that may cause GC problem
                 Key: CARBONDATA-3026
                 URL: https://issues.apache.org/jira/browse/CARBONDATA-3026
             Project: CarbonData
          Issue Type: Bug
          Components: data-load
            Reporter: xuchuanyin
            Assignee: xuchuanyin


During data loading, we will write some temp files (sort temp
files and temp fact data files) in some locations. In currently
implementation, we will add the locations to the CarbonProperties and
associated it with a special key that refers to the data loading.

After data loading, the temp locations are cleared, but the added
property is still remain in the CarbonProperties and never to be cleared.

This will cause the CarbonProperties object growing bigger and bigger
and lead to OOM problems if the thrift-server is a long time running
service. A local test shows that after adding different properties for
11 Billion times, the OOM happens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)