hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gopal V (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HIVE-20042) HiveServer2: All operations lock on a Single HiveConf object
Date Sun, 01 Jul 2018 06:00:01 GMT

     [ https://issues.apache.org/jira/browse/HIVE-20042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gopal V updated HIVE-20042:
---------------------------
    Labels: Concurrency  (was: )

> HiveServer2: All operations lock on a Single HiveConf object 
> -------------------------------------------------------------
>
>                 Key: HIVE-20042
>                 URL: https://issues.apache.org/jira/browse/HIVE-20042
>             Project: Hive
>          Issue Type: Bug
>          Components: HiveServer2
>    Affects Versions: 3.0.0
>            Reporter: Gopal V
>            Priority: Major
>              Labels: Concurrency
>
> With the 1000 user test, the session start/tear-down runs only at 100% CPU, which is
due to all threads locking on the same HiveConf object.
> OpenSession locks on 0x00000005c091a3a0
> {code}
> "HiveServer2-HttpHandler-Pool: Thread-65084" #65084 prio=5 os_prio=0 tid=0x00000000103bb000
nid=0x4a09 waiting for monitor entry [0x00007fc1b0987000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>     at org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418)
>     - waiting to lock <0x00000005c091a3a0> (a org.apache.hadoop.hive.conf.HiveConf)
>     at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711)
>     at org.apache.hadoop.conf.Configuration.get(Configuration.java:1437)
>     at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:4996)
>     at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5069)
>     at org.apache.hive.service.cli.thrift.ThriftCLIService.getUserName(ThriftCLIService.java:424)
>     at org.apache.hive.service.cli.thrift.ThriftCLIService.getSessionHandle(ThriftCLIService.java:467)
>     at org.apache.hive.service.cli.thrift.ThriftCLIService.OpenSession(ThriftCLIService.java:325)
> {code}
> GetOperationStatus locks on the same 
> {code}
> "HiveServer2-HttpHandler-Pool: Thread-65082" #65082 prio=5 os_prio=0 tid=0x00007fc2656be000
nid=0x4a06 waiting for monitor entry [0x00007fc3159db000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>     at org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418)
>     - waiting to lock <0x00000005c091a3a0> (a org.apache.hadoop.hive.conf.HiveConf)
>     at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711)
>     at org.apache.hadoop.conf.Configuration.get(Configuration.java:1437)
>     at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:4996)
>     at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:5069)
>     at org.apache.hive.service.cli.thrift.ThriftCLIService.GetOperationStatus(ThriftCLIService.java:709)
> {code}
> Session clean up locks on the same
> {code}
> "8fd1db09-9f96-49dc-becf-5702826bd4f5 HiveServer2-HttpHandler-Pool: Thread-64981" #64981
prio=5 os_prio=0 tid=0x000000001d1ab000 nid=0x23d5 waiting for monitor entry [0x00007fc1b65e3000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>     at org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418)
>     - waiting to lock <0x00000005c091a3a0> (a org.apache.hadoop.hive.conf.HiveConf)
>     at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711)
>     at org.apache.hadoop.conf.Configuration.get(Configuration.java:1177)
>     at org.apache.hadoop.conf.Configuration.getTrimmedStringCollection(Configuration.java:2122)
>     at org.apache.hadoop.hdfs.DFSUtilClient.getNameServiceIds(DFSUtilClient.java:197)
>     at org.apache.hadoop.hdfs.HAUtilClient.isLogicalUri(HAUtilClient.java:53)
> ...
>     at org.apache.hadoop.hdfs.DistributedFileSystem.delete(DistributedFileSystem.java:959)
>     at org.apache.hadoop.hive.ql.Context.clear(Context.java:724)
> {code}
> Hadoop RPC blocks on the same
> {code}
> "HiveServer2-HttpHandler-Pool: Thread-59227" #59227 prio=5 os_prio=0 tid=0x00007fc270aeb800
nid=0x129b waiting for monitor entry [0x00007fc28b7b5000]
>    java.lang.Thread.State: BLOCKED (on object monitor)
>     at org.apache.hadoop.conf.Configuration.getOverlay(Configuration.java:1418)
>     - waiting to lock <0x00000005c091a3a0> (a org.apache.hadoop.hive.conf.HiveConf)
>     at org.apache.hadoop.conf.Configuration.handleDeprecation(Configuration.java:711)
>     at org.apache.hadoop.conf.Configuration.get(Configuration.java:1177)
>     at org.apache.hadoop.conf.Configuration.getTrimmed(Configuration.java:1234)
>     at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:1459)
>     at org.apache.hadoop.ipc.Client$Connection.<init>(Client.java:451)
>     at org.apache.hadoop.ipc.Client.getConnection(Client.java:1532)
>  ...
>     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1580)
>     at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1734)
>     at org.apache.hadoop.fs.FileSystem.deleteOnExit(FileSystem.java:1677)
>     at org.apache.hadoop.hive.ql.session.SessionState.createPath(SessionState.java:791)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message