storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ankur Garg <ankurga...@gmail.com>
Subject Re: Sending Some Context when Failing a tuple.
Date Sun, 03 Jan 2016 19:00:27 GMT
Hi Ravi,

May be a very naive answer here but still posting it  :

I am assuming that once u catch the failed Tuple inside the fail method of
your spout  all you need to decide is whether this tuple should be replayed
or not .

I am assuming the object you are sending between spout and bolts are
already serializable .  How about adding this extra information fore
replaying the tuple to the same Object  (Since you already thinking of
storing it in some external storage I am assuming this info too is
serializable) . It may be a simple boolean flag too .

For ex :

Original Tuple u  r sending may be

OrigTuple implements Serializable
{
    ObjectA a;
    ObjectB b;
}

I am assuming a , b are all serializable or marked transient .

Now in case of failure you can attach Object C too which contains failure
information or simple boolean Flag which implies to the spout that it needs
to be played . For the ones which dont need to be played it takes default
value as false .


Like I said before , it is a very simple thought  but I could think of this
may work based on info u provided and assumptions I made.

Thanks
Ankur



On Sun, Jan 3, 2016 at 3:59 PM, Ravi Sharma <ping2ravi@gmail.com> wrote:

> Hi All,
> I would like to send some extra information back to spout when a tuple is
> failed in some Bolt, so that Spout can decide if it want it to replay or
> just put the message into queue outside storm for admins to view.
>
> So is there any way i can attach some more information when sending back
> failed tuple to spout.?
>
> One way i can think of is keeping such information outside storm in some
> datastore, with Tuple id and spout can lookup that, but looking for some
> way to do it via storm without bringing in other integration/datastore.
>
>
> Thanks
> Ravi.
>

Mime
View raw message