tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Siddharth Seth <sseth.apa...@gmail.com>
Subject Re: [jira] [Updated] (TEZ-2568) V_INPUT_DATA_INFORMATION may happen after vertex is initialized
Date Sat, 20 Jun 2015 02:21:04 GMT
Jeff. You may want to look at where this event is generated. Looked a bit when Vikram reported
it earlier. This happens when the deprecated awtParallelism Api is used. From a very brief
look - it looks like One of the VM callbacks - onRootInputInitialized? always sends this event
out in it's registered callback which it shouldn't do.



> On Jun 19, 2015, at 19:07, Jeff Zhang (JIRA) <jira@apache.org> wrote:
> 
> 
>     [ https://issues.apache.org/jira/browse/TEZ-2568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
> 
> Jeff Zhang updated TEZ-2568:
> ----------------------------
>    Description: 
> {code}
> 2015-06-19 15:57:28,462 ERROR [Dispatcher thread: Central] impl.VertexImpl: Can't handle
Invalid event V_INPUT_DATA_INFORMATION on vertex Map 2 with vertexId vertex_1434754502979_0002_2_00
at current state INITED
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_INPUT_DATA_INFORMATION
at INITED
>        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
>        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
>        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
>        at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1799)
>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:198)
>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1963)
>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1949)
>        at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
>        at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
>        at java.lang.Thread.run(Thread.java:722)
> {code}
> 
> Vertex move to INITED as long as its parallelism is determined, no null edges and root
inputs are initialized. RootInputDataInformation handling is not a precondition of vertex
move to INITED.   We can't wait for all the V_INPUT_DATA_INFORMATION events available in INITIALIZING
state, because it is not know how many V_INPUT_DATA_INFORMATION we may receive, it is determined
by VM.  So will allow V_INPUT_DATA_INFORMATION happens when vertex is initialized. 
> 
>  was:
> {code}
> 2015-06-19 15:57:28,462 ERROR [Dispatcher thread: Central] impl.VertexImpl: Can't handle
Invalid event V_INPUT_DATA_INFORMATION on vertex Map 2 with vertexId vertex_1434754502979_0002_2_00
at current state INITED
> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_INPUT_DATA_INFORMATION
at INITED
>        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
>        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
>        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
>        at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1799)
>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:198)
>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1963)
>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1949)
>        at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
>        at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
>        at java.lang.Thread.run(Thread.java:722)
> {code}
> 
> Vertex move to INITED as long as its parallelism is determined, no null edges and root
inputs are initialized.    We can't wait for all the V_INPUT_DATA_INFORMATION events available
in INITIALIZING state, because it is not know how many V_INPUT_DATA_INFORMATION we may receive,
it is determined by VM.  So will allow V_INPUT_DATA_INFORMATION happens when vertex is initialized.

> 
> 
>> V_INPUT_DATA_INFORMATION may happen after vertex is initialized
>> ---------------------------------------------------------------
>> 
>>                Key: TEZ-2568
>>                URL: https://issues.apache.org/jira/browse/TEZ-2568
>>            Project: Apache Tez
>>         Issue Type: Bug
>>           Reporter: Jeff Zhang
>>           Assignee: Jeff Zhang
>>        Attachments: a.log
>> 
>> 
>> {code}
>> 2015-06-19 15:57:28,462 ERROR [Dispatcher thread: Central] impl.VertexImpl: Can't
handle Invalid event V_INPUT_DATA_INFORMATION on vertex Map 2 with vertexId vertex_1434754502979_0002_2_00
at current state INITED
>> org.apache.hadoop.yarn.state.InvalidStateTransitonException: Invalid event: V_INPUT_DATA_INFORMATION
at INITED
>>        at org.apache.hadoop.yarn.state.StateMachineFactory.doTransition(StateMachineFactory.java:305)
>>        at org.apache.hadoop.yarn.state.StateMachineFactory.access$300(StateMachineFactory.java:46)
>>        at org.apache.hadoop.yarn.state.StateMachineFactory$InternalStateMachine.doTransition(StateMachineFactory.java:448)
>>        at org.apache.tez.state.StateMachineTez.doTransition(StateMachineTez.java:57)
>>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:1799)
>>        at org.apache.tez.dag.app.dag.impl.VertexImpl.handle(VertexImpl.java:198)
>>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1963)
>>        at org.apache.tez.dag.app.DAGAppMaster$VertexEventDispatcher.handle(DAGAppMaster.java:1949)
>>        at org.apache.tez.common.AsyncDispatcher.dispatch(AsyncDispatcher.java:183)
>>        at org.apache.tez.common.AsyncDispatcher$1.run(AsyncDispatcher.java:114)
>>        at java.lang.Thread.run(Thread.java:722)
>> {code}
>> Vertex move to INITED as long as its parallelism is determined, no null edges and
root inputs are initialized. RootInputDataInformation handling is not a precondition of vertex
move to INITED.   We can't wait for all the V_INPUT_DATA_INFORMATION events available in INITIALIZING
state, because it is not know how many V_INPUT_DATA_INFORMATION we may receive, it is determined
by VM.  So will allow V_INPUT_DATA_INFORMATION happens when vertex is initialized.
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v6.3.4#6332)

Mime
View raw message