spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Lemieux <david.lemi...@radialpoint.com>
Subject Re: Spark Streaming using Flume body size limitation
Date Fri, 23 May 2014 19:29:28 GMT
Created https://issues.apache.org/jira/browse/SPARK-1916
I'll submit a pull request soon.

/D
On May 23, 2014, at 9:56 AM, David Lemieux <david.lemieux@radialpoint.com> wrote:

> For some reason the patch did not make it.
>
> Trying via email:
>
>
>
> /D
>
> On May 23, 2014, at 9:52 AM, lemieud <david.lemieux@radialpoint.com> wrote:
>
> > Hi,
> >
> > I think I found the problem.
> > In SparkFlumeEvent the readExternal method use in.read(bodyBuff) which read
> > the first 1020 bytes, but no more. The code should make sure to read
> > everything.
> > The following change will fix the problem:
> > in.read(bodyBuff)
> > to:
> > in.readFully(bodyBuff)
> >
> > I attached a patch.
> >
> > If somebody can point me in the right direction, I'll gladly do the required
> > legwork to submit the patch upstream.
> >
> > Have a nice day.
> >
> > /D
> >
> >
> >
> > --
> > View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-Streaming-using-Flume-body-size-limitation-tp6127p6314.html
> > Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> *********************************************************************** This e-mail and
attachments are confidential, legally privileged, may be subject to copyright and sent solely
for the attention of the addressee(s). Any unauthorized use or disclosure is prohibited. Statements
and opinions expressed in this e-mail may not represent those of Radialpoint. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Le contenu du présent courriel est confidentiel, privilégié et peut être soumis à des
droits d'auteur. Il est envoyé à l'intention exclusive de son ou de ses destinataires. Il
est interdit de l'utiliser ou de le divulguer sans autorisation. Les opinions exprimées dans
le présent courriel peuvent diverger de celles de Radialpoint.
> <SparkFlumeStreamingBodySizeLimitationPath.diff>

*********************************************************************** This e-mail and attachments
are confidential, legally privileged, may be subject to copyright and sent solely for the
attention of the addressee(s). Any unauthorized use or disclosure is prohibited. Statements
and opinions expressed in this e-mail may not represent those of Radialpoint. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Le contenu du présent courriel est confidentiel, privilégié et peut être soumis à des
droits d'auteur. Il est envoyé à l'intention exclusive de son ou de ses destinataires. Il
est interdit de l'utiliser ou de le divulguer sans autorisation. Les opinions exprimées dans
le présent courriel peuvent diverger de celles de Radialpoint.

Mime
View raw message