[ https://issues.apache.org/jira/browse/TEZ-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jonathan Eagles resolved TEZ-3587.
----------------------------------
Resolution: Fixed
Fix Version/s: TEZ-3334
committed to feature branch TEZ-3334
> Fetcher fetchInputs() can NPE on srcAttempt due to missing entry in pathToAttemptMap
> ------------------------------------------------------------------------------------
>
> Key: TEZ-3587
> URL: https://issues.apache.org/jira/browse/TEZ-3587
> Project: Apache Tez
> Issue Type: Sub-task
> Reporter: Kuhu Shukla
> Assignee: Kuhu Shukla
> Fix For: TEZ-3334
>
> Attachments: TEZ-3587.001.patch
>
>
> With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to populate
the pathToAttemptsMap with the correct value of "partition" member of the PathPartition object.
This can cause an NPE.
> {code}
> [main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG diagnostics:
[Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02, diagnostics=[Task failed,
taskId=task_123_0001_2_02_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while
running task ( failure ) : java.lang.NullPointerException
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> , errorMessage=Fetch failed:java.lang.NullPointerException
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
> at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
|