tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jianfeng (Jeff) Zhang" <jzh...@hortonworks.com>
Subject Re: Question on Event
Date Tue, 01 Jul 2014 11:17:26 GMT
Hitesh/Bikas,

Thanks for your answering. It looks like event is used in 2 cases: tez
internal use and user usage. Some events can be used by users while some
events is only for internal usage ( Task Related event ).  And I think
currently tez don't support  customized events because Tez control how to
route the events and serialize/deserialize them (TezEvent). So the document
may should focus on the usage of the specific events rather than the event
interface ( maybe that's why the interface event is just empty)



Here's the table I make for the events (not completed ), let me know if
there's any thing wrong with it. Thanks

  *Event * *From * *To*   DataMovementEvent  Output  Input
InputFailedEvent  AM  Input   InputReadErrorEvent  Input  AM
RootInputDataInformationEvent  AM  Input   TaskAttemptCompletedEvent
TaskReporter  AM   TaskAttemptFailedEvent  TaskReporter  AM
TaskStatusUpdateEvent  TaskReporter  AM   VertexManagerEvent  Output  AM

Best Regards,
Jeff Zhang



On Sat, Jun 28, 2014 at 1:37 AM, Bikas Saha <bikas@hortonworks.com> wrote:

> Events are meant to be a common object that can carry a payload between
> framework->user (InputFailedEvent), user->framework (InputReadErrorEvent),
> user->user (DataMovementEvent).
>
> Events are routed based on event type and using internal metadata
> (source/destination metadata) present in the event. The metadata is
> automatically added by the system (eg. TezDAGChild adds source meta when
> the
> output produces an event). The main thing that users care about is the
> payload which is already serialized. The rest of the serde is taken care of
> by the system. This is by design so that the users don’t have to understand
> system internals and can use this easily. Their surface area of interaction
> is limited to sending/receiving the correct event/payload.
>
> The documentation per event can be improved to describe this better.
>
> Bikas
>
> -----Original Message-----
> From: Jianfeng (Jeff) Zhang [mailto:jzhang@hortonworks.com]
> Sent: Friday, June 27, 2014 12:28 AM
> To: dev@tez.incubator.apache.org
> Subject: Question on Event
>
> Hi all,
>
> I notice that class Event is used by both Input, Output, Processor and
> VertexManagerPlugin.
> But IMHO it's not clear for users on how to use them and user don't have
> full control on the events. Most of the logic is in tez-dag ( such as
> whether sending it to TezChild,  it need, How to serialize/ deserialize
> these events ) It's hard to add customized events, although it may be a
> rare
> requirement .
>
> I am thinking that in the Event class, we should provide some methods that
> allow user to specify these attributes ( may need other code change ). My
> purpose is to help user understand how to use these event. What's your
> thinking on this ? Thanks
>
>
>
> Best Regards,
> Jeff Zhang
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of
> this message is not the intended recipient, you are hereby notified that
> any
> printing, copying, dissemination, distribution, disclosure or forwarding of
> this communication is strictly prohibited. If you have received this
> communication in error, please contact the sender immediately and delete it
> from your system. Thank You.
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message