spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: How do I Process Streams that span multiple lines?
Date Tue, 04 Aug 2015 08:58:51 GMT
If you are using Kafka, then you can basically push an entire file as a
message to Kafka. In that case in your DStream, you will receive the single
message which is the contents of the file and it can of course span
multiple lines.

Thanks
Best Regards

On Mon, Aug 3, 2015 at 8:27 PM, Spark Enthusiast <sparkenthusiast@yahoo.in>
wrote:

> All  examples of Spark Stream programming that I see assume streams of
> lines that are then tokenised and acted upon (like the WordCount example).
>
> How do I process Streams that span multiple lines? Are there examples that
> I can use?
>

Mime
View raw message