hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <>
Subject [jira] [Commented] (HIVE-13176) OutOfMemoryError : GC overhead limit exceeded
Date Tue, 01 Mar 2016 00:07:18 GMT


Siddharth Seth commented on HIVE-13176:

There's several issues here, and two cases being looked at.

In the first case, there's a lot of failed queries. Driver instances are retained by the Shutdownhook
- 35K of them. Looking at the code, I couldn't see any direct leaks - and can only attribute
this to failed queries / connections - where the client may have failed to close the connection.

In the second case, there's two main culprits. DataNucleus PluginManager seems to be retaining
a large chunk of the heap. The second is a single FileSystem object which is over 100MB. This
is caused by retention of FSStatistics objects - and has been fixed in a Hadoop jira - we
should look at adding a workaround for this, i.e. consume the statistics instance as part
of the close. The second reason is a large number of entities registered in the deleteOnExit
tracker within FileSystem. That, I believe, is something that Hive will need to look at.

> OutOfMemoryError :  GC overhead limit exceeded
> ----------------------------------------------
>                 Key: HIVE-13176
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>            Reporter: Kavan Suresh
>            Assignee: Siddharth Seth
> Detected leaks while testing hiveserver2 concurrency setup with LLAP.
> 2016-02-26T12:50:58,131 ERROR [HiveServer2-Background-Pool: Thread-311030]: operation.Operation
( - Error running hive query:
> org.apache.hive.service.cli.HiveSQLException: Error while processing statement: FAILED:
Execution Error, return code -101 from org.apache.hadoop.hive.ql.exec.StatsTask. GC overhead
limit exceeded
>         at org.apache.hive.service.cli.operation.Operation.toSQLException(
>         at org.apache.hive.service.cli.operation.SQLOperation.runQuery(
>         at org.apache.hive.service.cli.operation.SQLOperation.access$100(
>         at org.apache.hive.service.cli.operation.SQLOperation$1$
>         at Method) ~[?:1.8.0_45]
>         at [?:1.8.0_45]
>         at
>         at org.apache.hive.service.cli.operation.SQLOperation$
>         at java.util.concurrent.Executors$ [?:1.8.0_45]
>         at [?:1.8.0_45]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
>         at java.util.concurrent.ThreadPoolExecutor$
>         at [?:1.8.0_45]

This message was sent by Atlassian JIRA

View raw message