hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20936) Allow the Worker thread in the metastore to run outside of it
Date Sat, 15 Dec 2018 03:01:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16721972#comment-16721972
] 

Hive QA commented on HIVE-20936:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  1s{color} |
{color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 27s{color} | {color:blue}
Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 29s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 42s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 57s{color}
| {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  2m 12s{color} | {color:blue}
standalone-metastore/metastore-common in master has 29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  1m  3s{color} | {color:blue}
standalone-metastore/metastore-server in master has 188 extant Findbugs warnings. {color}
|
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 42s{color} | {color:blue}
ql in master has 2310 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 37s{color} | {color:blue}
service in master has 48 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 26s{color} | {color:blue}
jdbc in master has 17 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 25s{color} | {color:blue}
hcatalog/streaming in master has 11 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 26s{color} | {color:blue}
streaming in master has 2 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 36s{color} | {color:blue}
itests/hive-unit in master has 2 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  3m 18s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 25s{color} | {color:blue}
Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  4m 40s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  3m 42s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  3m 42s{color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 41s{color} | {color:red}
ql: The patch generated 4 new + 640 unchanged - 7 fixed = 644 total (was 647) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 16s{color} | {color:red}
itests/hive-unit: The patch generated 1 new + 169 unchanged - 0 fixed = 170 total (was 169)
{color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 112 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  2s{color} | {color:red}
ql generated 3 new + 2309 unchanged - 1 fixed = 2312 total (was 2310) {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m 54s{color} | {color:red}
standalone-metastore_metastore-common generated 1 new + 16 unchanged - 0 fixed = 17 total
(was 16) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 12s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 53m 15s{color} | {color:black}
{color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
|  |  Field MetaStoreCompactorThread.threadId masks field in superclass org.apache.hadoop.hive.ql.txn.compactor.CompactorThread
 In MetaStoreCompactorThread.java:superclass org.apache.hadoop.hive.ql.txn.compactor.CompactorThread
 In MetaStoreCompactorThread.java |
|  |  Field MetaStoreCompactorThread.rs masks field in superclass org.apache.hadoop.hive.ql.txn.compactor.CompactorThread
 In MetaStoreCompactorThread.java:superclass org.apache.hadoop.hive.ql.txn.compactor.CompactorThread
 In MetaStoreCompactorThread.java |
|  |  Null passed for non-null parameter of RemoteCompactorThread.resolveTable(CompactionInfo)
in org.apache.hadoop.hive.ql.txn.compactor.Worker.run()  Method invoked at Worker.java:of
RemoteCompactorThread.resolveTable(CompactionInfo) in org.apache.hadoop.hive.ql.txn.compactor.Worker.run()
 Method invoked at Worker.java:[line 116] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03)
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-15336/dev-support/hive-personality.sh
|
| git revision | master / 4e41560 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus/diff-checkstyle-ql.txt
|
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus/diff-checkstyle-itests_hive-unit.txt
|
| whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus/whitespace-eol.txt
|
| findbugs | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus/new-findbugs-ql.html
|
| javadoc | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus/diff-javadoc-javadoc-standalone-metastore_metastore-common.txt
|
| modules | C: standalone-metastore/metastore-common standalone-metastore/metastore-server
ql service jdbc hcatalog/streaming streaming itests/hive-unit U: . |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-15336/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> Allow the Worker thread in the metastore to run outside of it
> -------------------------------------------------------------
>
>                 Key: HIVE-20936
>                 URL: https://issues.apache.org/jira/browse/HIVE-20936
>             Project: Hive
>          Issue Type: Improvement
>          Components: Transactions
>            Reporter: Jaume M
>            Assignee: Jaume M
>            Priority: Major
>         Attachments: HIVE-20936.1.patch, HIVE-20936.10.patch, HIVE-20936.11.patch, HIVE-20936.2.patch,
HIVE-20936.3.patch, HIVE-20936.4.patch, HIVE-20936.5.patch, HIVE-20936.6.patch, HIVE-20936.7.patch,
HIVE-20936.8.patch, HIVE-20936.8.patch
>
>
> Currently the Worker thread in the metastore in bounded to the metastore, mainly because
of the TxnHandler that it has. This thread runs some map reduce jobs which may not only be
an option wherever the metastore is running. A solution for this can be to run this thread
in HS2 depending on a flag.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message