storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From I PVP <i...@hotmail.com>
Subject Re: how to keep storm running when console ends
Date Fri, 04 Aug 2017 15:34:34 GMT
Hi J. R. Pauley.

If that helps :  I use supervisord not only for Storm but also for other solutions within
the ecosystem.

- IP VP


On August 4, 2017 at 12:29:41 PM, Bobby Evans (evans@yahoo-inc.com<mailto:evans@yahoo-inc.com>)
wrote:

Java by default also tries to keep running if something unexpected happens.  If an exception
is not caught and goes all the way up the stack, that thread will just print out a stack trace
to stderr and then quietly stop running.  Do you know how many zombie daemons I have had to
debug/fix because something slightly unexpected happened on a critical thread.  I personally
added in a generic uncaught exception handler to all of the Hadoop daemons years ago because
it was happening all of the time in the early days of YARN.  Storm is almost 100% immune to
those situations, and has been since the beginning.

That is not to say that storm is not stable.  On most of our clusters the only time nimbus
or a supervisor goes down is when we upgrade it.  We have had daemons running for months.
 And if they do go down they typically recover in a few seconds, and my team gets an alert
so we can debug what happened.  Running storm (and I would argue any critical daemon) under
supervision for production environments is just a best practice.

For debugging/testing something then go ahead and use nohup.

- Bobby



On Friday, August 4, 2017, 9:43:10 AM CDT, J.R. Pauley <jrpauley@gmail.com> wrote:


thanks for all the responses. screen and nohup seem easy to adopt.

I guess for something intended to run on large clusters I'm surprised at the default behavior
of storm. It seems quite the opposite of what I would've guessed.

I also have a Heron instance running in similar fashion and it's default behavior is to keep
running which seems more what I expected.

On Fri, Aug 4, 2017 at 9:28 AM, M. Aaron Bossert <mabossert@gmail.com<mailto:mabossert@gmail.com>>
wrote:
Try adding nohup at the beginning of each of those commands.  Nohup prevents a process from
terminating when it's parent shell is closed.

nohup /opt/storm/apache*/bin/ storm nimbus &


Sent from my iPhone

On Aug 4, 2017, at 09:08, Ethan Li <ethanli@yahoo-inc.com<mailto:ethanli@yahoo-inc.com>>
wrote:

Hi,

Storm docs<https://github.com/apache/storm/blob/43fde85e350c7feaa8e289fab0eedcfafedab44d/docs/Command-line-client.md#nimbus>
recommend to use daemontools or monit which provides error recovery and etc..

I use linux "nohup" or "screen" command for simplicity.

Ethan




On Friday, August 4, 2017 7:29 AM, J.R. Pauley <jrpauley@gmail.com<mailto:jrpauley@gmail.com>>
wrote:


this seems silly but I have not figured out how to keep nimbus, supervisor running after console
session ends. zookeeper survives but supervisor and nimbus shut down when the console session
ends.

I have storm 1.0.2 installed under /opt/storm and starting up as:
/opt/storm/apache*/bin/storm nimbus&
/opt/storm/apache*/bin/storm supervisor&
/opt/storm/apache*/bin/storm ui&
/opt/storm/apache*/bin/storm drpc&

I know I can add a while loop to keep the console active but there has to be a better way



Mime
View raw message