spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lemieud <david.lemi...@radialpoint.com>
Subject Re: Spark Streaming using Flume body size limitation
Date Fri, 23 May 2014 13:52:38 GMT
Hi,

I think I found the problem.
In SparkFlumeEvent the readExternal method use in.read(bodyBuff) which read
the first 1020 bytes, but no more. The code should make sure to read
everything.
The following change will fix the problem:
in.read(bodyBuff)
to:
in.readFully(bodyBuff)

I attached a patch.

If somebody can point me in the right direction, I'll gladly do the required
legwork to submit the patch upstream.

Have a nice day.

/D



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-using-Flume-body-size-limitation-tp6127p6314.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message