storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexis Sukrieh <>
Subject Re: How to recover from a Spout excpetion?
Date Fri, 23 May 2014 05:31:44 GMT
Thanks Cody, patching the ScopesImpl class is indeed the root solution,
I'll see if I can do that.

Meanwhile, is there a way for me to automatically kill/resubmit my topology
when I encounter this error?

​I realized that exiting when an exception is caught is not enough, the
workers are restarted but the topology remains in a bogus state.

2014-05-22 23:32 GMT+02:00 Cody A. Ray <>:

> I think the problem is that the code bundled with your topology is
> serialized by the nimbus and transferred to the supervisors, so all the
> code you use needs to be serializable. Something in twitter4j must be a
> using a non-serializable class (the ScopesImpl class).
> I don't have a suggestion other than patch/contribute twitter4j or look at
> a different twitter library... But I think that's the root cause.
> Someone else may have a better suggestion.
> -Cody
> On May 22, 2014 1:00 PM, "Alexis Sukrieh" <> wrote:
>> Hi,
>> I have built a topology with a spout that connects to Twitter's streaming
>> API (with twitter4j), I have also 4 bolts that count words inside.
>> This is mainly a proof of concept for me, in order to get up to speed
>> with Storm.
>> So far, it's working fine... until my Spout gets an exception from
>> twitter4j:
>> java.lang.RuntimeException: java.lang.RuntimeException:
>> When this happens, my topology is somehow broken: the Redis database that gets updated
by the last bolt is not updated anymore.
>> All I can do is killing the topology and submit it again. That's a bit annoying as
I'd like my topology to never stops running.
>> I'd like to understand how to catch this exception, and make sure my topology is
protected against invalid tuples.
>> In the Spout class, I really don't see how I could catch the exception.
>> Any help is more than welcome.
>> Thanks.
>> --
>> Alexis Sukrieh

Alexis Sukrieh -

"People get annoyed when you try to debug them." -- Larry Wall

View raw message