hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-17657) export/import for MM tables is broken
Date Sun, 22 Apr 2018 06:36:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-17657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447117#comment-16447117
] 

Hive QA commented on HIVE-17657:
--------------------------------

| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  0s{color} | {color:blue}
Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  0s{color} |
{color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m  9s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 12s{color} |
{color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 45s{color}
| {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  6s{color} |
{color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 40s{color}
| {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 16s{color} |
{color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 16s{color} | {color:green}
the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 46s{color} | {color:red}
ql: The patch generated 36 new + 597 unchanged - 13 fixed = 633 total (was 610) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  0s{color} | {color:red}
The patch has 4 line(s) that end in whitespace. Use git apply --whitespace=fix <<patch_file>>.
Refer https://git-scm.com/docs/git-apply {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  5s{color} |
{color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 16s{color}
| {color:green} The patch does not generate ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 42s{color} | {color:black}
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1+deb8u1 (2016-09-03)
x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus_PreCommit-HIVE-Build-10401/dev-support/hive-personality.sh
|
| git revision | master / 84481cf |
| Default Java | 1.8.0_111 |
| checkstyle | http://104.198.109.242/logs//PreCommit-HIVE-Build-10401/yetus/diff-checkstyle-ql.txt
|
| whitespace | http://104.198.109.242/logs//PreCommit-HIVE-Build-10401/yetus/whitespace-eol.txt
|
| modules | C: ql U: ql |
| Console output | http://104.198.109.242/logs//PreCommit-HIVE-Build-10401/yetus.txt |
| Powered by | Apache Yetus    http://yetus.apache.org |


This message was automatically generated.



> export/import for MM tables is broken
> -------------------------------------
>
>                 Key: HIVE-17657
>                 URL: https://issues.apache.org/jira/browse/HIVE-17657
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Transactions
>            Reporter: Eugene Koifman
>            Assignee: Sergey Shelukhin
>            Priority: Major
>              Labels: mm-gap-2
>         Attachments: HIVE-17657.01.patch, HIVE-17657.02.patch, HIVE-17657.patch
>
>
> there is mm_exim.q but it's not clear from the tests what file structure it creates 
> On import the txnids in the directory names would have to be remapped if importing to
a different cluster.  Perhaps export can be smart and export highest base_x and accretive
deltas (minus aborted ones).  Then import can ...?  It would have to remap txn ids from the
archive to new txn ids.  This would then mean that import is made up of several transactions
rather than 1 atomic op.  (all locks must belong to a transaction)
> One possibility is to open a new txn for each dir in the archive (where start/end txn
of file name is the same) and commit all of them at once (need new TMgr API for that).  This
assumes using a shared lock (if any!) and thus allows other inserts (not related to import)
to occur.
> What if you have delta_6_9, such as a result of concatenate?  If we stipulate that this
must mean that there is no delta_6_6 or any other "obsolete" delta in the archive we can map
it to a new single txn delta_x_x.
> Add read_only mode for tables (useful in general, may be needed for upgrade etc) and
use that to make the above atomic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message