tez-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: Question on Event
Date Fri, 27 Jun 2014 17:37:31 GMT
Events are meant to be a common object that can carry a payload between
framework->user (InputFailedEvent), user->framework (InputReadErrorEvent),
user->user (DataMovementEvent).

Events are routed based on event type and using internal metadata
(source/destination metadata) present in the event. The metadata is
automatically added by the system (eg. TezDAGChild adds source meta when the
output produces an event). The main thing that users care about is the
payload which is already serialized. The rest of the serde is taken care of
by the system. This is by design so that the users don’t have to understand
system internals and can use this easily. Their surface area of interaction
is limited to sending/receiving the correct event/payload.

The documentation per event can be improved to describe this better.

Bikas

-----Original Message-----
From: Jianfeng (Jeff) Zhang [mailto:jzhang@hortonworks.com]
Sent: Friday, June 27, 2014 12:28 AM
To: dev@tez.incubator.apache.org
Subject: Question on Event

Hi all,

I notice that class Event is used by both Input, Output, Processor and
VertexManagerPlugin.
But IMHO it's not clear for users on how to use them and user don't have
full control on the events. Most of the logic is in tez-dag ( such as
whether sending it to TezChild,  it need, How to serialize/ deserialize
these events ) It's hard to add customized events, although it may be a rare
requirement .

I am thinking that in the Event class, we should provide some methods that
allow user to specify these attributes ( may need other code change ). My
purpose is to help user understand how to use these event. What's your
thinking on this ? Thanks



Best Regards,
Jeff Zhang

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader of
this message is not the intended recipient, you are hereby notified that any
printing, copying, dissemination, distribution, disclosure or forwarding of
this communication is strictly prohibited. If you have received this
communication in error, please contact the sender immediately and delete it
from your system. Thank You.

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Mime
View raw message