hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load
Date Mon, 25 Jun 2018 08:56:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16522048#comment-16522048
] 

Hive QA commented on HIVE-19924:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928918/HIVE-19924.01.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/12086/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12086/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12086/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ date '+%Y-%m-%d %T.%3N'
2018-06-25 08:53:50.165
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-12086/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-06-25 08:53:50.168
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 2277661 HIVE-19922: TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
(Peter Vary, reviewed by Jason Dere)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 2277661 HIVE-19922: TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
(Peter Vary, reviewed by Jason Dere)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-06-25 08:53:50.782
+ rm -rf ../yetus_PreCommit-HIVE-Build-12086
+ mkdir ../yetus_PreCommit-HIVE-Build-12086
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-12086
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-12086/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
error: a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestJdbcDriver2.java: does not
exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/QueryState.java: does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/Task.java: does not exist in index
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/ReplicationSemanticAnalyzer.java: does
not exist in index
error: a/service/src/java/org/apache/hive/service/cli/operation/OperationManager.java: does
not exist in index
error: a/service/src/java/org/apache/hive/service/cli/operation/SQLOperation.java: does not
exist in index
error: a/service/src/java/org/apache/hive/service/cli/session/HiveSession.java: does not exist
in index
error: a/service/src/java/org/apache/hive/service/cli/session/HiveSessionImpl.java: does not
exist in index
error: a/service/src/java/org/apache/hive/service/server/KillQueryImpl.java: does not exist
in index
Going to apply patch with: git apply -p1
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven
protoc-jar: executing: [/tmp/protoc8452245142444521116.exe, --version]
libprotoc 2.5.0
protoc-jar: executing: [/tmp/protoc8452245142444521116.exe, -I/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore,
--java_out=/data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources,
/data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/protobuf/org/apache/hadoop/hive/metastore/metastore.proto]
ANTLR Parser Generator  Version 3.5.2
Output file /data/hiveptest/working/apache-github-source-source/standalone-metastore/target/generated-sources/org/apache/hadoop/hive/metastore/parser/FilterParser.java
does not exist: must build /data/hiveptest/working/apache-github-source-source/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/parser/Filter.g
org/apache/hadoop/hive/metastore/parser/Filter.g
[ERROR] COMPILATION ERROR : 
[ERROR] /data/hiveptest/working/apache-github-source-source/serde/src/test/org/apache/hadoop/hive/serde2/io/TestTimestampWritable.java:[46,8]
class TestTimestampWritableV2 is public, should be declared in a file named TestTimestampWritableV2.java
[ERROR] /data/hiveptest/working/apache-github-source-source/serde/src/test/org/apache/hadoop/hive/serde2/io/TestDateWritable.java:[48,8]
class TestDateWritableV2 is public, should be declared in a file named TestDateWritableV2.java
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.6.1:testCompile
(default-testCompile) on project hive-serde: Compilation failure: Compilation failure:
[ERROR] /data/hiveptest/working/apache-github-source-source/serde/src/test/org/apache/hadoop/hive/serde2/io/TestTimestampWritable.java:[46,8]
class TestTimestampWritableV2 is public, should be declared in a file named TestTimestampWritableV2.java
[ERROR] /data/hiveptest/working/apache-github-source-source/serde/src/test/org/apache/hadoop/hive/serde2/io/TestDateWritable.java:[48,8]
class TestDateWritableV2 is public, should be declared in a file named TestDateWritableV2.java
[ERROR] -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn <goals> -rf :hive-serde
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-12086
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928918 - PreCommit-HIVE-Build

> Tag distcp jobs run by Repl Load
> --------------------------------
>
>                 Key: HIVE-19924
>                 URL: https://issues.apache.org/jira/browse/HIVE-19924
>             Project: Hive
>          Issue Type: Task
>          Components: repl
>    Affects Versions: 3.1.0, 4.0.0
>            Reporter: mahesh kumar behera
>            Assignee: mahesh kumar behera
>            Priority: Major
>             Fix For: 3.1.0, 4.0.0
>
>         Attachments: HIVE-19924.01.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will allow hive
to kill these jobs in case beacon retries, or hs2 dies and beacon issues a kill command.
>  * one of the tags should definitely be the query_id that starts the job : With this
flow beacon before retrying the bootstrap load, will issue a kill command to hs2 with the
query id of the previous issued command. hs2 will then kill an running jobs on yarn tagged
with the Query_id.
>  * To get around the additional failure point as mentioned above. The jobs can be tagged
with an additional unique tag_id provided by Beacon in the WITH clause in repl load command
to be used to tag distcp jobs ). Enhance the kill api to take the tag as input and kill jobs
associated with that tag. Problem here is how do we validate the association of the tag with
a hive query id to make sure this api is not used to kill jobs run by other components, however
we can provide this capability to only admins and should be ok in that case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message