tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Eagles (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (TEZ-3587) Fetcher fetchInputs() can NPE on srcAttempt due to missing entry in pathToAttemptMap
Date Thu, 19 Jan 2017 23:12:26 GMT

     [ https://issues.apache.org/jira/browse/TEZ-3587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Eagles resolved TEZ-3587.
----------------------------------
       Resolution: Fixed
    Fix Version/s: TEZ-3334

committed to feature branch TEZ-3334

> Fetcher fetchInputs() can NPE on srcAttempt due to missing entry in pathToAttemptMap
> ------------------------------------------------------------------------------------
>
>                 Key: TEZ-3587
>                 URL: https://issues.apache.org/jira/browse/TEZ-3587
>             Project: Apache Tez
>          Issue Type: Sub-task
>            Reporter: Kuhu Shukla
>            Assignee: Kuhu Shukla
>             Fix For: TEZ-3334
>
>         Attachments: TEZ-3587.001.patch
>
>
> With Tez Shuffle Handler (or composite fetch on), unordered fetcher can fail to populate
the pathToAttemptsMap with the correct value of "partition" member of the PathPartition object.
This can cause an NPE.
> {code}
> [main] examples.HashJoinExample (TezExampleBase.java:runDag(190)) - DAG diagnostics:
[Vertex failed, vertexName=joiner, vertexId=vertex_123_0001_2_02, diagnostics=[Task failed,
taskId=task_123_0001_2_02_000001, diagnostics=[TaskAttempt 0 failed, info=[Error: Error while
running task ( failure ) : java.lang.NullPointerException
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
>         at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> , errorMessage=Fetch failed:java.lang.NullPointerException
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.fetchInputs(Fetcher.java:893)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:597)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.doHttpFetch(Fetcher.java:486)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:284)
>         at org.apache.tez.runtime.library.common.shuffle.Fetcher.callInternal(Fetcher.java:76)
>         at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
>         at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message