hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20349) Implement Retry Logic in HiveDruidSplit for Scan Queries
Date Fri, 17 Aug 2018 19:10:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584291#comment-16584291
] 

Hive QA commented on HIVE-20349:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  1s{color} |
{color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 42s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 56s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 23s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 51s{color}
| {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 27s{color} | {color:blue}
druid-handler in master has 13 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 54s{color} | {color:blue}
ql in master has 2307 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 16s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  9s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m  1s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 24s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 24s{color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 11s{color} | {color:red}
druid-handler: The patch generated 21 new + 30 unchanged - 2 fixed = 51 total (was 32) {color}
|
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m  0s{color}
| {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 52s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 10s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 13s{color} | {color:red}
The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 53s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03)
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-13306/dev-support/hive-personality.sh
|
| git revision | master / 5681647 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-13306/yetus/diff-checkstyle-druid-handler.txt
|
| asflicense | http://104.198.109.242/logs//PreCommit-HIVE-Build-13306/yetus/patch-asflicense-problems.txt
|
| modules | C: druid-handler ql U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-13306/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> Implement Retry Logic in HiveDruidSplit for Scan Queries
> --------------------------------------------------------
>
>                 Key: HIVE-20349
>                 URL: https://issues.apache.org/jira/browse/HIVE-20349
>             Project: Hive
>          Issue Type: Bug
>          Components: Druid integration
>            Reporter: Nishant Bangarwa
>            Assignee: Nishant Bangarwa
>            Priority: Major
>         Attachments: HIVE-20349.patch
>
>
> while distributing druid scan query we check where the segments are loaded and then each
HiveDruidSplit directly queries the historical node. 
> There are few cases when we need to retry and refetch the segments. 
> # The segment is loaded on multiple historical nodes and one of them went down. in this
case when we do not get response from one segment, we query the next replica. 
> # The segment was loaded onto a realtime task and was handed over, when we query the
realtime task has already finished. In this case there is no replica. The Split needs to query
the broker again for the location of the segment and then send the query to correct historical
node. 
> This is also the root cause of failure of druidkafkamini_basic.q test, where the segment
handover happens before the scan query is executed.
> Note: This is not a problem when we are directly querying Druid brokers as the broker
handles the retry logic. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message