spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Fregly <ch...@fregly.com>
Subject Re: network wordcount example
Date Tue, 01 Apr 2014 01:13:03 GMT
@eric-

i saw this exact issue recently while working on the KinesisWordCount.

are you passing "local[2]" to your example as the MASTER arg versus just
"local" or "local[1]"?

you need at least 2.  it's documented as "n>1" in the scala source docs -
which is easy to mistake for n>=1.

i just ran the NetworkWordCount sample and confirmed that local[1] does not
work, but  local[2] does work.

give that a whirl.

-chris




On Mon, Mar 31, 2014 at 10:41 AM, Diana Carroll <dcarroll@cloudera.com>wrote:

> Not sure what data you are sending in.  You could try calling
> "lines.print()" instead which should just output everything that comes in
> on the stream.  Just to test that your socket is receiving what you think
> you are sending.
>
>
> On Mon, Mar 31, 2014 at 12:18 PM, eric perler <ericperler@hotmail.com>wrote:
>
>> Hello
>>
>> i just started working with spark today... and i am trying to run the
>> wordcount network example
>>
>> i created a socket server and client.. and i am sending data to the
>> server in an infinite loop
>>
>> when i run the spark class.. i see this output in the console...
>>
>> -------------------------------------------
>> Time: 1396281891000 ms
>> -------------------------------------------
>>
>> 14/03/31 11:04:51 INFO SparkContext: Job finished: take at
>> DStream.scala:586, took 0.056794606 s
>> 14/03/31 11:04:51 INFO JobScheduler: Finished job streaming job
>> 1396281891000 ms.0 from job set of time 1396281891000 ms
>> 14/03/31 11:04:51 INFO JobScheduler: Total delay: 0.101 s for time
>> 1396281891000 ms (execution: 0.058 s)
>> 14/03/31 11:04:51 INFO TaskSchedulerImpl: Remove TaskSet 3.0 from pool
>>
>> but i dont see any output from the workcount operation when i make this
>> call...
>>
>> wordCounts.print();
>>
>> any help is greatly appreciated
>>
>> thanks in advance
>>
>
>

Mime
View raw message