tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xin Yang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (TEZ-3779) Tez OutOfMemoryError: Java heap space
Date Tue, 27 Jun 2017 17:45:01 GMT
Xin Yang created TEZ-3779:
-----------------------------

             Summary: Tez OutOfMemoryError: Java heap space
                 Key: TEZ-3779
                 URL: https://issues.apache.org/jira/browse/TEZ-3779
             Project: Apache Tez
          Issue Type: Bug
    Affects Versions: 0.8.5
            Reporter: Xin Yang


Query:

{code:java}
select  a11.ISSR_CTRY_CD  CTRY_CD,
        a14.DMSTC_INTL_IND  DMSTC_INTL_IND,
        a11.ISSR_USR_BUS_ID  bus_id,
        ' '  CustCol_73,
        a11.CPD_MNTH_ID  CPD_MONTH_ID,
        a11.prod_afs_cd_vcis  prod_acct_fund_srce_cd_vcis,
        sum((Case when a13.card_prsnt_cd in (1) then a11.auth_tran_us_amt else NULL end))
 AUTHTRANAMTUSD,
        sum((Case when a13.card_prsnt_cd in (1) then a11.CS_TRAN_CNT else NULL end))  AUTHTRANCNT,
        (Case when max((Case when a13.card_prsnt_cd in (1) then 1 else 0 end)) = 1 then count(distinct
(Case when a13.card_prsnt_cd in (1) then a11.pymt_crd_acct_num_norm else NULL end)) else NULL
end)  WJXBFS1,
        max((Case when a13.card_prsnt_cd in (1) then 1 else 0 end))  GODWFLAG1_1,
        sum((Case when a13.card_prsnt_cd in (0) then a11.auth_tran_us_amt else NULL end))
 AUTHTRANAMTUSD1,
        sum((Case when a13.card_prsnt_cd in (0) then a11.CS_TRAN_CNT else NULL end))  AUTHTRANCNT1,
        (Case when max((Case when a13.card_prsnt_cd in (0) then 1 else 0 end)) = 1 then count(distinct
(Case when a13.card_prsnt_cd in (0) then a11.pymt_crd_acct_num_norm else NULL end)) else NULL
end)  WJXBFS2,
        max((Case when a13.card_prsnt_cd in (0) then 1 else 0 end))  GODWFLAG4_1
from    opebi_bi.tcaef_auth_dtl_h       a11
        join    OPCODE.TEDC_ECI_MOTO    a12
          on    (a11.ECI_MOTO_CD = a12.ECI_MOTO_CD)
        join    OPCODE.TEDC_CARD_PRSNT_EBI      a13
          on    (a11.POS_ENTRY_MODE_CD = a13.POS_ENTRY_MODE_CD and 
        a11.POS_ENV_CD = a13.POS_ENV_CD and 
        a12.eci_moto_grp_cd = a13.eci_moto_grp_cd)
        join    OPCODE.TEDC_ACCT_MRCH_JRSDCTN_CD        a14
          on    (a11.VCIS_ACCT_MRCH_JRSDCTN_CD = a14.ACCT_MRCH_JRSDCTN_CD)
        join    OPCODE.TEDC_GLBL_PROD_ID        a15
          on    (a11.ALP_ACCT_PROD_ID = a15.PROD_ID_CD)
        join    OPCODE.TEDC_AUTH_RESP_CD        a16
          on    (a11.resp_cd = a16.AUTH_RESP_CD)
where   (a11.MRCH_CATG_CD not in (6010, 6011)
 and a11.CPD_MNTH_ID BETWEEN 201602 and 201602
 and a11.PROC_TRAN_CD in ('00')
 and a11.ISSR_CTRY_CD in (76)
 and a11.reqst_msg_typ_cd in ('0100', '0200')
 and a16.AUTH_RESP_RLUP_CD in (0, 1, 4, 5)
 and a11.resp_cd not in ('13', '--')
 and a11.reqst_msg_typ_cd in ('0100', '0200', '0000')
 and a11.stip_advc_cd in ('1', '2', '3', '4', '5', '6')
 and a11.ACQR_BIN_NUM not in (746922)
 and a15.PROD_BRND_CD in ('VISA')
 and a15.PROD_ID_PLTFRM_CD in ('BZ', ' ', 'CN', 'GV', 'CO')
 and a11.acqr_pcr_num not in ('8088', '9088')
 and (a13.card_prsnt_cd in (1)
 or a13.card_prsnt_cd in (0)))
group by        a11.ISSR_CTRY_CD,
        a14.DMSTC_INTL_IND,
        a11.ISSR_USR_BUS_ID,
        a11.CPD_MNTH_ID,
        a11.prod_afs_cd_vcis;
{code}


Stacktrace:


{code:java}
Status: Failed
Vertex failed, vertexName=Map 3, vertexId=vertex_1495595408051_21107_2_03, diagnostics=[Task
failed, taskId=task_1495595408051_21107_2_03_000000, diagnostics=[TaskAttempt 0 failed, info=[Error:
exceptio
nThrown=java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56)
        at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46)
        at org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput.<init>(MemoryFetchedInput.java:38)
        at org.apache.tez.runtime.library.common.shuffle.impl.SimpleFetchedInputAllocator.allocate(SimpleFetchedInputAllocator.java:141)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:717)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:489)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:398)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:195)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:70)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
, errorMessage=Fetch failed:java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:56)
        at org.apache.hadoop.io.BoundedByteArrayOutputStream.<init>(BoundedByteArrayOutputStream.java:46)
        at org.apache.tez.runtime.library.common.shuffle.MemoryFetchedInput.<init>(MemoryFetchedInput.java:38)
        at org.apache.tez.runtime.library.common.shuffle.impl.SimpleFetchedInputAllocator.allocate(SimpleFetchedInputAllocator.java:141)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:717)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:489)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:398)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:195)
        at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:70)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
], TaskAttempt 1 failed, info=[Error: Failure while running task:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:229)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:147)
        ... 14 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.util.concurrent.ExecutionException:
java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:388)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:378)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:481)
        at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:438)
        at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:375)
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:214)
        ... 15 more
Caused by: java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap
space
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.hadoop.hive.ql.exec.Operator.completeInitialization(Operator.java:386)
        ... 20 more
Caused by: java.lang.OutOfMemoryError: Java heap space
        at org.apache.hadoop.hive.serde2.WriteBuffers.nextBufferToWrite(WriteBuffers.java:241)
        at org.apache.hadoop.hive.serde2.WriteBuffers.write(WriteBuffers.java:217)
        at org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer$LazyBinaryKvWriter.writeKey(MapJoinBytesTableContainer.java:235)
        at org.apache.hadoop.hive.ql.exec.persistence.BytesBytesMultiHashMap.put(BytesBytesMultiHashMap.java:445)
        at org.apache.hadoop.hive.ql.exec.persistence.MapJoinBytesTableContainer.putRow(MapJoinBytesTableContainer.java:365)
        at org.apache.hadoop.hive.ql.exec.tez.HashTableLoader.load(HashTableLoader.java:191)
        at org.apache.hadoop.hive.ql.exec.MapJoinOperator.loadHashTable(MapJoinOperator.java:288)
        at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:173)
        at org.apache.hadoop.hive.ql.exec.MapJoinOperator$1.call(MapJoinOperator.java:169)
        at org.apache.hadoop.hive.ql.exec.tez.ObjectCache.retrieve(ObjectCache.java:75)
        at org.apache.hadoop.hive.ql.exec.tez.ObjectCache$1.call(ObjectCache.java:92)
        ... 4 more
], TaskAttempt 2 failed, info=[Error: Failure while running task:java.lang.RuntimeException:
java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:171)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:137)
        at org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:337)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:179)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable$1.run(TezTaskRunner.java:171)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:422)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1642)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:171)
        at org.apache.tez.runtime.task.TezTaskRunner$TaskRunnerCallable.callInternal(TezTaskRunner.java:167)
        at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.RuntimeException: Map operator initialization failed
        at org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.init(MapRecordProcessor.java:229)
        at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:147)
        ... 14 more
]], Vertex did not succeed due to OWN_TASK_FAILURE, failedTasks:1 killedTasks:0, Vertex vertex_1495595408051_21107_2_03
[Map 3] killed/failed due to:null]Vertex killed, vertexName=Reducer 7, vertexId=ve
rtex_1495595408051_21107_2_06, diagnostics=[Vertex received Kill while in RUNNING state.,
Vertex did not succeed due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:2, Vertex vertex_1495595408051_211
07_2_06 [Reducer 7] killed/failed due to:null]Vertex killed, vertexName=Map 6, vertexId=vertex_1495595408051_21107_2_05,
diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed
 due to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1495595408051_21107_2_05
[Map 6] killed/failed due to:null]Vertex killed, vertexName=Map 5, vertexId=vertex_1495595408051_21107_2
_04, diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due
to OTHER_VERTEX_FAILURE, failedTasks:0 killedTasks:1, Vertex vertex_1495595408051_21107_2_04
[Map 5] killed/fai
led due to:null]Vertex killed, vertexName=Map 1, vertexId=vertex_1495595408051_21107_2_02,
diagnostics=[Vertex received Kill while in RUNNING state., Vertex did not succeed due to OTHER_VERTEX_FAILURE,

failedTasks:0 killedTasks:41, Vertex vertex_1495595408051_21107_2_02 [Map 1] killed/failed
due to:null]DAG did not succeed due to VERTEX_FAILURE. failedVertices:1 killedVertices:4
{code}




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message