spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From anshu shukla <>
Subject Creating topology in spark streaming
Date Wed, 06 May 2015 07:53:35 GMT
Please help  guys, Even  After going through all the examples given i have
not understood how to pass the  D-streams  from one bolt/logic to other
(without writing it on HDFS etc.) just like emit function in storm .
Suppose i have topology with 3  bolts(say)

*BOLT1(parse the tweets nd emit tweet using given
hashtags)=====>Bolt2(Complex logic for sentiment analysis over
tweets)=======>BOLT3(submit tweets to the sql database using spark SQL)*

Now  since Sentiment analysis will take most of the time ,we have to
increase its level of parallelism for tuning latency. Howe to increase the
levele of parallelism since the logic of topology is not clear .

Thanks & Regards,
Anshu Shukla
Indian Institute of Sciences

View raw message