spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rapelly kartheek <>
Subject rsync problem
Date Fri, 19 Sep 2014 08:02:46 GMT

I'd made some modifications to  the spark source code in the master and
reflected them to the slaves using rsync.

I followed this command:
rsync -avL --progress path/to/spark-1.0.0  username@destinationhostname

This worked perfectly. But, I wanted to simultaneously rsync all the
slaves. So, added the other slaves as following:

rsync -avL --progress path/to/spark-1.0.0  username@destinationhostname
:path/to/destdirectory username@slave2:path username@slave3:path and so on.

But this didn't work. Anyway, for now, I did it individually for each node.

Can someone give me the right syntax.

Secondly, after this rsync, I find that my cluster has become tremendously
Sometimes the cluster is just shutting down. Job execution is not happening.

Can someone throw some light on this aspect.

thank you

View raw message