tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: Question on Event
Date Tue, 01 Jul 2014 16:25:16 GMT
The internal events that derive from base class yarn.AbstractEvent are an
internal implementation detail. These are not user facing.

The ones that matter to the user are o.a.t.runtime.api.event.

Bikas

-----Original Message-----
From: Jianfeng (Jeff) Zhang [mailto:jzhang@hortonworks.com]
Sent: Tuesday, July 01, 2014 4:17 AM
To: dev@tez.incubator.apache.org
Subject: Re: Question on Event

Hitesh/Bikas,

Thanks for your answering. It looks like event is used in 2 cases: tez
internal use and user usage. Some events can be used by users while some
events is only for internal usage ( Task Related event ).  And I think
currently tez don't support  customized events because Tez control how to
route the events and serialize/deserialize them (TezEvent). So the document
may should focus on the usage of the specific events rather than the event
interface ( maybe that's why the interface event is just empty)



Here's the table I make for the events (not completed ), let me know if
there's any thing wrong with it. Thanks

  *Event * *From * *To*   DataMovementEvent  Output  Input
InputFailedEvent  AM  Input   InputReadErrorEvent  Input  AM
RootInputDataInformationEvent  AM  Input   TaskAttemptCompletedEvent
TaskReporter  AM   TaskAttemptFailedEvent  TaskReporter  AM
TaskStatusUpdateEvent  TaskReporter  AM   VertexManagerEvent  Output  AM

Best Regards,
Jeff Zhang



On Sat, Jun 28, 2014 at 1:37 AM, Bikas Saha <bikas@hortonworks.com> wrote:

> Events are meant to be a common object that can carry a payload
> between
> framework->user (InputFailedEvent), user->framework
> framework->(InputReadErrorEvent),
> user->user (DataMovementEvent).
>
> Events are routed based on event type and using internal metadata
> (source/destination metadata) present in the event. The metadata is
> automatically added by the system (eg. TezDAGChild adds source meta
> when the output produces an event). The main thing that users care
> about is the payload which is already serialized. The rest of the
> serde is taken care of by the system. This is by design so that the
> users don’t have to understand system internals and can use this
> easily. Their surface area of interaction is limited to
> sending/receiving the correct event/payload.
>
> The documentation per event can be improved to describe this better.
>
> Bikas
>
> -----Original Message-----
> From: Jianfeng (Jeff) Zhang [mailto:jzhang@hortonworks.com]
> Sent: Friday, June 27, 2014 12:28 AM
> To: dev@tez.incubator.apache.org
> Subject: Question on Event
>
> Hi all,
>
> I notice that class Event is used by both Input, Output, Processor and
> VertexManagerPlugin.
> But IMHO it's not clear for users on how to use them and user don't
> have full control on the events. Most of the logic is in tez-dag (
> such as whether sending it to TezChild,  it need, How to serialize/
> deserialize these events ) It's hard to add customized events,
> although it may be a rare requirement .
>
> I am thinking that in the Event class, we should provide some methods
> that allow user to specify these attributes ( may need other code
> change ). My purpose is to help user understand how to use these
> event. What's your thinking on this ? Thanks
>
>
>
> Best Regards,
> Jeff Zhang
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
> entity to which it is addressed and may contain information that is
> confidential, privileged and exempt from disclosure under applicable
> law. If the reader of this message is not the intended recipient, you
> are hereby notified that any printing, copying, dissemination,
> distribution, disclosure or forwarding of this communication is
> strictly prohibited. If you have received this communication in error,
> please contact the sender immediately and delete it from your system.
> Thank You.
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or
> entity to which it is addressed and may contain information that is
> confidential, privileged and exempt from disclosure under applicable
> law. If the reader of this message is not the intended recipient, you
> are hereby notified that any printing, copying, dissemination,
> distribution, disclosure or forwarding of this communication is
> strictly prohibited. If you have received this communication in error,
> please contact the sender immediately and delete it from your system.
> Thank You.
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message