hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Koifman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-20719) SELECT statement fails after UPDATE with hive.optimize.sort.dynamic.partition optimization and vectorization on
Date Thu, 11 Oct 2018 18:01:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-20719?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16646834#comment-16646834 ] 

Eugene Koifman commented on HIVE-20719:
---------------------------------------

Disabling vectorization doesn't actually fix it.  The difference is that Vectorized acid reader has an assert to make sure the file name and ROW__IDs in the file match, non-vectorized doesn't (just produces bad data by not reading all relevant delete events).

Here is what the state looks like on disk after the update statement.  Everything from the update stmt is written to bucket_0.  This creates data skew for insert events and will ignore delete events that apply other buckets at read time.

{noformat}
acid_uap/ds=tomorrow/delta_0000001_0000001_0000/bucket_00001 [length: 702]
{"operation":0,"originalTransaction":1,"bucket":536936448,"rowId":0,"currentTransaction":1,"row":{"a":1,"b":"bah"}}
________________________________________________________________________________________________________________________

acid_uap/ds=tomorrow/delta_0000001_0000001_0000/bucket_00000 [length: 703]
{"operation":0,"originalTransaction":1,"bucket":536870912,"rowId":0,"currentTransaction":1,"row":{"a":2,"b":"yah"}}
________________________________________________________________________________________________________________________

acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00001 [length: 711]
{"operation":0,"originalTransaction":2,"bucket":536936448,"rowId":0,"currentTransaction":2,"row":{"a":1,"b":"bah"}}
________________________________________________________________________________________________________________________

acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000 [length: 695]
{"operation":0,"originalTransaction":2,"bucket":536870912,"rowId":0,"currentTransaction":2,"row":{"a":2,"b":"yah"}}
________________________________________________________________________________________________________________________

acid_uap/ds=tomorrow/delta_0000003_0000003_0000/bucket_00000 [length: 733]
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":{"a":2,"b":"fred"}}
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":1,"currentTransaction":3,"row":{"a":1,"b":"fred"}}
________________________________________________________________________________________________________________________

acid_uap/ds=today/delta_0000003_0000003_0000/bucket_00000 [length: 733]
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":{"a":2,"b":"fred"}}
{"operation":0,"originalTransaction":3,"bucket":536870912,"rowId":1,"currentTransaction":3,"row":{"a":1,"b":"fred"}}
________________________________________________________________________________________________________________________

acid_uap/ds=tomorrow/delete_delta_0000003_0000003_0000/bucket_00000 [length: 713]
{"operation":2,"originalTransaction":1,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":null}
{"operation":2,"originalTransaction":1,"bucket":536936448,"rowId":0,"currentTransaction":3,"row":null}
________________________________________________________________________________________________________________________

acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000 [length: 705]
{"operation":2,"originalTransaction":2,"bucket":536870912,"rowId":0,"currentTransaction":3,"row":null}
{"operation":2,"originalTransaction":2,"bucket":536936448,"rowId":0,"currentTransaction":3,"row":null}
{noformat}

> SELECT statement fails after UPDATE with hive.optimize.sort.dynamic.partition optimization and vectorization on
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-20719
>                 URL: https://issues.apache.org/jira/browse/HIVE-20719
>             Project: Hive
>          Issue Type: Bug
>          Components: Transactions, Vectorization
>    Affects Versions: 4.0.0
>            Reporter: Vineet Garg
>            Assignee: Eugene Koifman
>            Priority: Major
>
> *Reproducer*
> {code:sql}
>  set hive.optimize.sort.dynamic.partition=true ;
> create table acid_uap(a int, b varchar(128)) partitioned by (ds string) clustered by (a) into 2 buckets stored as orc TBLPROPERTIES ('transactional'='true');
> insert into table acid_uap partition (ds='tomorrow') values (1, 'bah'),(2, 'yah') ;
> insert into table acid_uap partition (ds='today') values (1, 'bah'),(2, 'yah') ;
> select a,b,ds from acid_uap order by a,b;
> update acid_uap set b = 'fred';
> select a,b,ds from acid_uap order by a,b;
> {code}
> *Error*
> {code:java}
> Status: Failed
> Vertex failed, vertexName=Map 1, vertexId=vertex_1539123809352_0001_5_00, diagnostics=[Task failed, taskId=task_1539123809352_0001_5_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_1:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_2:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_3:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1539123809352_0001_5_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]
> Vertex killed, vertexName=Reducer 2, vertexId=vertex_1539123809352_0001_5_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1539123809352_0001_5_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]
> DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
> FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 1, vertexId=vertex_1539123809352_0001_5_00, diagnostics=[Task failed, taskId=task_1539123809352_0001_5_00_000000, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_0:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 1 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_1:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 2 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_2:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ], TaskAttempt 3 failed, info=[Error: Error while running task ( failure ) : attempt_1539123809352_0001_5_00_000000_3:java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> 	at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> 	at org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> 	at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> 	at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> 	at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:206)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:152)
> 	at org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:68)
> 	at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:426)
> 	at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> 	... 16 more
> Caused by: java.io.IOException: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderCreationException(HiveIOExceptionHandlerChain.java:97)
> 	at org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderCreationException(HiveIOExceptionHandlerUtil.java:57)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:419)
> 	at org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.initNextRecordReader(TezGroupedSplitsInputFormat.java:203)
> 	... 21 more
> Caused by: java.io.IOException: Corrupted records with different bucket ids from the containing bucket file found! Expected bucket id 0, however found DeleteRecordKey(2,536936448(1.1.0),0).  (OrcSplit [file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delta_0000002_0000002_0000/bucket_00000, start=3, length=361, isOriginal=false, fileLength=798, hasFooter=false, hasBase=true, deltas=2],file:/Users/vgarg/hive_temp/vgarg/hive/warehouse/dp_sort.db/acid_uap/ds=today/delete_delta_0000003_0000003_0000/bucket_00000)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.checkBucketId(VectorizedOrcAcidRowBatchReader.java:1441)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry$DeleteReaderValue.next(VectorizedOrcAcidRowBatchReader.java:1362)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.readAllDeleteEventsFromDeleteDeltas(VectorizedOrcAcidRowBatchReader.java:1644)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader$ColumnizedDeleteEventRegistry.<init>(VectorizedOrcAcidRowBatchReader.java:1567)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:243)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:138)
> 	at org.apache.hadoop.hive.ql.io.orc.VectorizedOrcAcidRowBatchReader.<init>(VectorizedOrcAcidRowBatchReader.java:133)
> 	at org.apache.hadoop.hive.ql.io.orc.OrcInputFormat.getRecordReader(OrcInputFormat.java:2003)
> 	at org.apache.hadoop.hive.ql.io.HiveInputFormat.getRecordReader(HiveInputFormat.java:416)
> 	... 22 more
> ]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1539123809352_0001_5_00 [Map 1] killed/failed due to:OWN_TASK_FAILURE]Vertex killed, vertexName=Reducer 2, vertexId=vertex_1539123809352_0001_5_01, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1539123809352_0001_5_01 [Reducer 2] killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:1
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message