storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From clay teahouse <clayteaho...@gmail.com>
Subject Re: bolt stop receiving tuples
Date Thu, 04 Dec 2014 13:13:06 GMT
Hello,
I did comment out the entire execute body, except for a log statement, but
the issue persists. The method is never visited. Just noticed if I replace
the remote consumer, with a local one, the issue goes away and all the
tuples are consumed properly. I don't see anything in the storm log
indicating having issue with the remote consumer.
In both cases (remote and local consumers), Bolt B writes to a socket.
Here is my topology config:
        config.setNumWorkers(2);
        config.put(Config.TOPOLOGY_WORKER_SHARED_THREAD_POOL_SIZE, 4);
        config.put(Config.TOPOLOGY_TRANSFER_BUFFER_SIZE, 32768);
        config.put(Config.TOPOLOGY_EXECUTOR_RECEIVE_BUFFER_SIZE, 32768);
        config.put(Config.TOPOLOGY_RECEIVER_BUFFER_SIZE, 16384);
        config.put(Config.TOPOLOGY_EXECUTOR_SEND_BUFFER_SIZE, 32768);
        config.put(Config.TOPOLOGY_DEBUG, true);

thanks,
Clay

On Thu, Dec 4, 2014 at 6:22 AM, Devang Shah <devangshah97@gmail.com> wrote:

> Can you post your topology configuration here like the no. Of workers, no.
> of instances of each spout/bolt, max pending spout etc.
>
> What processing are you doing in the bolt B - connecting to external
> service ? Try replacing all the code in execute method of bolt B with a log
> statement and check if it's still an issue.
>
> Thanks and Regards,
> Devang
> On 4 Dec 2014 19:28, "clay teahouse" <clayteahouse@gmail.com> wrote:
>
>> This is a local cluster. I don't see anything interesting in the logs
>> that would tell me anything.  I even removed Bolt A from the picture
>> (meaning Spout->BoltB), and still Bolt B hangs, after the first pull. If
>> that helps, the complete config is:
>>
>> spout-> Bolt B -> the remote non-storm entity that Bolt B sends data to.
>>
>> The interesting thing is that every time I restart the topology, one more
>> tuple (of the backlog) is sent to the remote entity, and then the
>> everything stops. So, if I restart the topology enough times (and the spout
>> doesn't consume any new data), the remote server will ultimately get all
>> the old tuples. It seems the tuples are buffered  and sent one by one to
>> the remote entity, when the topology restarts.
>>
>> -Clay
>>
>> On Thu, Dec 4, 2014 at 3:25 AM, Vladi Feigin <vladif86@gmail.com> wrote:
>>
>>> Usually in such case you should start from looking the logs : supervisor
>>> and worker
>>>
>>> On Wed, Dec 3, 2014 at 6:09 PM, clay teahouse <clayteahouse@gmail.com>
>>> wrote:
>>>
>>>> Hello All,
>>>>
>>>> I have this configuration:
>>>>
>>>> spout -> Bolt A (emits tuples) -> Bolt B
>>>>
>>>> Bolt A emits tuples successfully but bolt B stops receiving tuples
>>>> after the first time (it never enters the execute after the first time).
>>>> The first time execution seems to be successful. Any idea what the issue
>>>> could be or how trouble shoot the issue?
>>>>
>>>> thanks,
>>>> Clay
>>>>
>>>>
>>>>
>>>
>>

Mime
View raw message