hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <>
Subject [jira] [Commented] (HIVE-12528) don't start HS2 Tez sessions in a single thread
Date Fri, 11 Dec 2015 05:23:11 GMT


Siddharth Seth commented on HIVE-12528:

+    if (threadCount <= 1) {
+      for (int i = 0; i < blockingQueueLength; i++) {
+        // The queue is FIFO, so if we cycle thru length items, we'd start each session once.
+        startNextSessionFromQueue();
+      }
Replace with threadCount == 1 ?, and a precondition for the thredCount to not be below 1 ?

The threads end up throwing a RuntimeException in case of an Error. This would otherwise have
been caught by the HiveServer2 static initialization block. Is it now relying upon the default
uncaught exception handler ? Would be better to propagate the exception upwards as it's done
today (and maybe a CompletionService / ListeningExecutor with Callbacks)

     *  with this the ordering of sessions in the queue will be (with 2 sessions 3 queues)
     *  s1q1, s1q2, s1q3, s2q1, s2q2, s2q3 there by ensuring uniform distribution of
     *  the sessions across queues at least to begin with. Then as sessions get freed up,
the list
     *  may change this ordering.
This statement no longer stands. Given that it's only for the the first set of jobs anyway
- I don't think this is a problem. cc [~vikram.dixit]

I'm not sure how the thread safety - specifically visibility aspects are handled (both before
and after the patch). SessionStates, TezClient instances etc are created in a single thread
(now multiple threads), and then used in completely different threads. What is guaranteeing
correct visibility ?

> don't start HS2 Tez sessions in a single thread
> -----------------------------------------------
>                 Key: HIVE-12528
>                 URL:
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Sergey Shelukhin
>         Attachments: HIVE-12528.patch
> Starting sessions in parallel would improve the startup time.

This message was sent by Atlassian JIRA

View raw message