hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12528) don't start HS2 Tez sessions in a single thread
Date Fri, 11 Dec 2015 05:23:11 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052192#comment-15052192
] 

Siddharth Seth commented on HIVE-12528:
---------------------------------------

{code}
+    if (threadCount <= 1) {
+      for (int i = 0; i < blockingQueueLength; i++) {
+        // The queue is FIFO, so if we cycle thru length items, we'd start each session once.
+        startNextSessionFromQueue();
+      }
{code}
Replace with threadCount == 1 ?, and a precondition for the thredCount to not be below 1 ?

The threads end up throwing a RuntimeException in case of an Error. This would otherwise have
been caught by the HiveServer2 static initialization block. Is it now relying upon the default
uncaught exception handler ? Would be better to propagate the exception upwards as it's done
today (and maybe a CompletionService / ListeningExecutor with Callbacks)

{code}
/*
     *  with this the ordering of sessions in the queue will be (with 2 sessions 3 queues)
     *  s1q1, s1q2, s1q3, s2q1, s2q2, s2q3 there by ensuring uniform distribution of
     *  the sessions across queues at least to begin with. Then as sessions get freed up,
the list
     *  may change this ordering.
     */
{code}
This statement no longer stands. Given that it's only for the the first set of jobs anyway
- I don't think this is a problem. cc [~vikram.dixit]


I'm not sure how the thread safety - specifically visibility aspects are handled (both before
and after the patch). SessionStates, TezClient instances etc are created in a single thread
(now multiple threads), and then used in completely different threads. What is guaranteeing
correct visibility ?


> don't start HS2 Tez sessions in a single thread
> -----------------------------------------------
>
>                 Key: HIVE-12528
>                 URL: https://issues.apache.org/jira/browse/HIVE-12528
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-12528.patch
>
>
> Starting sessions in parallel would improve the startup time.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message