hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karen Coppage (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (HIVE-24858) UDFClassLoader leak in Configuration.CACHE_CLASSES
Date Tue, 06 Jul 2021 09:07:00 GMT

     [ https://issues.apache.org/jira/browse/HIVE-24858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Karen Coppage resolved HIVE-24858.
----------------------------------
    Fix Version/s: 4.0.0
       Resolution: Fixed

Committed to master on March 10. Thanks for reviewing [~pvary]!

> UDFClassLoader leak in Configuration.CACHE_CLASSES
> --------------------------------------------------
>
>                 Key: HIVE-24858
>                 URL: https://issues.apache.org/jira/browse/HIVE-24858
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Karen Coppage
>            Assignee: Karen Coppage
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: 4.0.0
>
>          Time Spent: 20m
>  Remaining Estimate: 0h
>
> If a UDF jar has been registered in a session and a temporary function created from it,
when the session is closed its UDFClassLoader is not GC'd as it has been leaked to the session's
HiveConf object's cache. Since the ClassLoader is not GC'd, the UDF jar's classes aren't GC'd
from Metaspace. This can potentially lead to Metaspace OOM.
>  Path to GC root is:
> {code:java}
> Class Name                                                                          
                                               | Shallow Heap | Retained Heap
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
> contextClassLoader org.apache.hive.service.server.ThreadWithGarbageCleanup @ 0x7164deb50
 HiveServer2-Handler-Pool: Thread-72 Thread|          128 |        79,072
> referent java.util.WeakHashMap$Entry @ 0x7164e67d0                                  
                                               |           40 |           824
> '- [6] java.util.WeakHashMap$Entry[16] @ 0x71581aac0                                
                                               |           80 |         5,056
>    '- table java.util.WeakHashMap @ 0x71580f510                                     
                                               |           48 |         6,920
>       '- CACHE_CLASSES class org.apache.hadoop.conf.Configuration @ 0x71580f3d8     
                                               |           64 |        74,528
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message