storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mostafa Gomaa <mgo...@trendak.com>
Subject Re: ShellBolt dies frequently
Date Sun, 26 Feb 2017 09:10:58 GMT
Hi Jungtaek,

I did set the topology.subprocess.timeout.secs value to a non-default
value, and it all seems to work perfectly now. Thanks a bunch

On Thu, Feb 23, 2017 at 5:51 PM, Jungtaek Lim <kabhwan@gmail.com> wrote:
> Please try to change log level to DEBUG, and see heartbeat for ShellBolt is
> updated correctly.
>
> In ShellBolt, unless subprocess takes numerous time to handle a tuple,
> heartbeat timeout should not happen. If you are not able to fix the
> heartbeat issue, you can set topology.subprocess.timeout.secs to quite large
> value so that heartbeat of ShellBolt never timed out.
>
> Thanks,
> Jungtaek Lim (HeartSaVioR)
>
> 2017년 2월 24일 (금) 오전 12:45, Kunal Gupta <kunal.gupta@kartrocket.com>님이
작성:
>>
>> Can you put your code in try and except.
>>
>> Create a logger and call log. info within the function of bolt.
>>
>> On 23 Feb 2017 21:10, "Mostafa Gomaa" <mgomaa@trendak.com> wrote:
>>>
>>> The code in the bolt itself is not crashing, any ideas?
>>>
>>> On Feb 23, 2017 5:35 PM, "Kunal Gupta" <kunal.gupta@kartrocket.com>
>>> wrote:
>>>>
>>>> Have you enabled a logs of your PyBolt. py?
>>>>
>>>> I think your code of bolt is getting crashed.
>>>>
>>>> On 23 Feb 2017 20:59, "Mostafa Gomaa" <mgomaa@trendak.com> wrote:
>>>>>
>>>>> Hello all,
>>>>>
>>>>> I am trying out adding a python bolt in my topology, but it keeps
>>>>> crashing quite frequently. Before adding this python bolt, my topology
>>>>> was EXTREMELY stable and would run for months without any issues.
>>>>> However, after adding the python bolt, the following exception happens
>>>>> quite frequently:
>>>>>
>>>>> 2017-02-23 15:14:04.080 o.a.s.t.ShellBolt [ERROR] Halting process:
>>>>> ShellBolt died. Command: [python, PyBolt.py], ProcessInfo pid:31162,
>>>>> name:PythonBolt exitCode:-1, errorString:
>>>>> java.lang.RuntimeException: subprocess heartbeat timeout
>>>>> at
>>>>> org.apache.storm.task.ShellBolt$BoltHeartbeatTimerTask.run(ShellBolt.java:322)
>>>>> [storm-core-1.0.2.jar:1.0.2]
>>>>> at
>>>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>>>> [?:1.8.0_121]
>>>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>>>>> [?:1.8.0_121]
>>>>> at
>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>>>>> [?:1.8.0_121]
>>>>> at
>>>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>>>>> [?:1.8.0_121]
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>>>> [?:1.8.0_121]
>>>>> at
>>>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>>>> [?:1.8.0_121]
>>>>> at java.lang.Thread.run(Thread.java:745) [?:1.8.0_121]
>>>>> 2017-02-23 15:14:04.092 o.a.s.d.executor [ERROR]
>>>>>
>>>>>
>>>>> I am using storm 1.0.2. Any tips or ideas?
>>>>>
>>>>> Thanks

Mime
View raw message