hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Prasanth Jayachandran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14397) Queries launched after reopening of tez session launches additional sessions
Date Mon, 01 Aug 2016 18:31:20 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14397?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15402582#comment-15402582
] 

Prasanth Jayachandran commented on HIVE-14397:
----------------------------------------------

[~sershe] The root cause of this bug is this code
https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java#L275-L287

If the queue name is set in conf then it always creates a new session when getSession is invoked.


The part that fixes this is conf.unset("tez.queue.name"). This makes sure we use the sessions
from the pool when available. 

The other part of the patch
{code}
conf.set(TezConfiguration.TEZ_QUEUE_NAME, sessionState.getQueueName());
{code}
is required to reopen the initial sessions when user specified queue name is not available.
If we don't set this then all the reopened sessions will use "default" queue. 


> Queries launched after reopening of tez session launches additional sessions
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-14397
>                 URL: https://issues.apache.org/jira/browse/HIVE-14397
>             Project: Hive
>          Issue Type: Bug
>          Components: Tez
>    Affects Versions: 2.1.0, 2.2.0
>            Reporter: Takahiko Saito
>            Assignee: Prasanth Jayachandran
>            Priority: Critical
>         Attachments: HIVE-14397.1.patch
>
>
> Say we have configured hive.server2.tez.default.queues with 2 queues q1 and q2 with default
expiry interval of 5 mins.
> After 5 mins of non-usage the sessions corresponding to queues q1 and q2 will be expired.
When new set of queries are issue after this expiry, the default sessions backed by q1 and
q2 and reopened again. Now when we run more queries the reopened sessions are not used instead
new session is opened. 
> At this point there will be 4 sessions running (2 abandoned sessions and 2 current sessions).




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message