spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Percy <mpe...@apache.org>
Subject Re: Spark streaming on YARN?
Date Tue, 21 Jan 2014 20:13:34 GMT
Just FYI I got it working in standalone mode with no probs, so thanks for
the help Tathagata. I was not able to get it working on YARN, I gave up.

Thanks,
Mike


On Fri, Jan 10, 2014 at 6:17 PM, Tathagata Das
<tathagata.das1565@gmail.com>wrote:

> Let me know if you have any trouble getting it running on the standalone
> mode. That will be easier for us to help out with.
>
> TD
>
>
> On Fri, Jan 10, 2014 at 5:47 PM, Mike Percy <mpercy@apache.org> wrote:
>
>> On Fri, Jan 10, 2014 at 6:35 AM, Neal Sidhwaney <nealsid@gmail.com>wrote:
>>
>>> Are you setting the SPARK_JAVA_OPTS environment variable? Under 0.8.1,
>>> we had to set that to -Dlog4j.configuration=<path to logging properties
>>> file>  to get STDOUT for workers.  This was in standalone mode, though
>>> not with YARN, so YMMV.
>>>  Neal
>>>
>>
>> Thanks for the idea Neal. The docs say that just putting it into conf/
>> should do the trick, but apparently that's not the case. I tried setting
>> that and it just stopped printing on the shell and had no effect on the
>> deployed container.
>>
>> Too much struggling with this, I'm going to work on getting this running
>> in standalone mode.
>>
>> Mike
>>
>>
>

Mime
View raw message