spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashic Mahtab <>
Subject Starting a spark streaming app in init.d
Date Fri, 23 Jan 2015 20:27:48 GMT
I'm trying to kick off a spark streaming job to a stand alone master using spark submit inside
of init.d. This is what I have:

DAEMON="spark-submit --class Streamer --executor-memory 500M --total-executor-cores 4 /path/to/assembly.jar"

start() {
        $DAEMON -p /var/run/ &
        echo "OK" &&
        return 0

However, will return 0 even if spark_submit fails. Is there a way to run spark-submit in the
background and return 0 only if it successfully starts up? Or better yet, is there something
in spark-submit that will allow me to do this, perhaps via a command line argument?

View raw message