spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ayan guha <guha.a...@gmail.com>
Subject Re: Creating topology in spark streaming
Date Wed, 06 May 2015 08:06:51 GMT
Every transformation on a dstream will create another dstream. You may want
to take a look at foreachrdd? Also, kindly share your code so people can
help better
On 6 May 2015 17:54, "anshu shukla" <anshushukla0@gmail.com> wrote:

> Please help  guys, Even  After going through all the examples given i have
> not understood how to pass the  D-streams  from one bolt/logic to other
> (without writing it on HDFS etc.) just like emit function in storm .
> Suppose i have topology with 3  bolts(say)
>
> *BOLT1(parse the tweets nd emit tweet using given
> hashtags)=====>Bolt2(Complex logic for sentiment analysis over
> tweets)=======>BOLT3(submit tweets to the sql database using spark SQL)*
>
>
> Now  since Sentiment analysis will take most of the time ,we have to
> increase its level of parallelism for tuning latency. Howe to increase the
> levele of parallelism since the logic of topology is not clear .
>
> --
> Thanks & Regards,
> Anshu Shukla
> Indian Institute of Sciences
>

Mime
View raw message