hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-21891) Break up DDLTask - cleanup
Date Thu, 20 Jun 2019 00:09:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-21891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16868132#comment-16868132
] 

Hive QA commented on HIVE-21891:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12972266/HIVE-21891.04.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/17661/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/17661/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-17661/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit status 1 and
output '+ date '+%Y-%m-%d %T.%3N'
2019-06-20 00:07:47.067
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-17661/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2019-06-20 00:07:47.071
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at cd42db4 HIVE-21892: Trusted domain authentication should look at X-Forwarded-For
header as well (Prasanth Jayachandran reviewed by Jason Dere, Ashutosh Bapat)
+ git clean -f -d
Removing standalone-metastore/metastore-server/src/gen/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at cd42db4 HIVE-21892: Trusted domain authentication should look at X-Forwarded-For
header as well (Prasanth Jayachandran reviewed by Jason Dere, Ashutosh Bapat)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2019-06-20 00:07:48.990
+ rm -rf ../yetus_PreCommit-HIVE-Build-17661
+ mkdir ../yetus_PreCommit-HIVE-Build-17661
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-17661
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-17661/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh /data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:7982: trailing whitespace.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.ddl.DDLTask. Table tstsrc
is not locked 
/data/hiveptest/working/scratch/build.patch:7992: trailing whitespace.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.ddl.DDLTask. Table tstsrcpart
is not locked 
/data/hiveptest/working/scratch/build.patch:8022: trailing whitespace.
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.ddl.DDLTask. Database
lockneg1 is not locked 
warning: 3 lines add whitespace errors.
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q -Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] The build could not read 1 project -> [Help 1]
[ERROR]   
[ERROR]   The project  (/data/hiveptest/working/apache-github-source-source/ql/pom.xml) has
1 error
[ERROR]     Non-parseable POM /data/hiveptest/working/apache-github-source-source/ql/pom.xml:
end tag name </depencency> must be the same as start tag <dependency> from line
760 (position: TEXT seen ...</version>\n    </depencency>... @764:18)  @ line
764, column 18 -> [Help 2]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following
articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/ModelParseException
+ result=1
+ '[' 1 -ne 0 ']'
+ rm -rf yetus_PreCommit-HIVE-Build-17661
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12972266 - PreCommit-HIVE-Build

> Break up DDLTask - cleanup
> --------------------------
>
>                 Key: HIVE-21891
>                 URL: https://issues.apache.org/jira/browse/HIVE-21891
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Hive
>    Affects Versions: 3.1.1
>            Reporter: Miklos Gergely
>            Assignee: Miklos Gergely
>            Priority: Major
>              Labels: refactor-ddl
>             Fix For: 4.0.0
>
>         Attachments: HIVE-21891.01.patch, HIVE-21891.02.patch, HIVE-21891.03.patch, HIVE-21891.04.patch
>
>
> DDLTask was a huge class, more than 5000 lines long. The related DDLWork was also a huge
class, which had a field for each DDL operation it supported. The goal was to refactor these
in order to have everything cut into more handleable classes under the package  org.apache.hadoop.hive.ql.exec.ddl:
>  * have a separate class for each operation
>  * have a package for each operation group (database ddl, table ddl, etc), so the amount
of classes under a package is more manageable
>  * make all the requests (DDLDesc subclasses) immutable - most of them are now
>  * DDLTask should be agnostic to the actual operations
>  * right now let's ignore the issue of having some operations handled by DDLTask which
are not actual DDL operations (lock, unlock, desc...)
> In the interim time when there were two DDLTask and DDLWork classes in the code base
the new ones in the new package were called DDLTask2 and DDLWork2 thus avoiding the usage
of fully qualified class names where both the old and the new classes were in use.
> Step #12: rename DDLTask2 and DDLWork2, now that they are alone. Remove the old DDLDesc.
Instead of registering, now DDLTask finds the DDLOperations, and registers them itself.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message