nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Mangold <csm9...@gmail.com>
Subject Re: PutJMS performance reading from NIFI Queue
Date Wed, 20 May 2015 20:44:29 GMT
Just going straight from the GetFile or GenerateFlowFile processor (via a
NIFI queue) to PutJMS.

I guess I have to read up on FlowFile penalization, I am not familiar with
the concept.

Chris



On Wed, May 20, 2015 at 3:52 PM, Mark Payne <markap14@hotmail.com> wrote:

> Ohh, I totally misunderstood the problem. I thought you were saying that
> you were pushing messages at 30 per second.
>
> When you say you have a GetFile or a GenerateFlowFile feeding the PutJMS,
> are you feeding directly from those processors, or is there something in
> between?
>
> I'm wondering if the FlowFiles are possibly penalized?
>
> Thanks
> -Mark
>
> ________________________________
> > Date: Wed, 20 May 2015 15:44:37 -0400
> > Subject: Re: PutJMS performance reading from NIFI Queue
> > From: csm9105@gmail.com
> > To: users@nifi.incubator.apache.org
> >
> > Mark:
> >
> > I have been varying the "Message Batch Size" between 10 and 1000
> > with no appreciable improvement in performance. I notice a larger
> > batch of messages going into ActiveMQ but the Processor only is getting
> > triggered every ~ 30 seconds.
> >
> > Chris
> >
> > On Wed, May 20, 2015 at 2:50 PM, Mark Payne
> > <markap14@hotmail.com<mailto:markap14@hotmail.com>> wrote:
> > Chris,
> >
> > What value do you have the "Batch Size" property set to? That can make
> > a really big difference in throughput.
> >
> > ________________________________
> >> Date: Wed, 20 May 2015 13:56:46 -0400
> >> Subject: PutJMS performance reading from NIFI Queue
> >> From: csm9105@gmail.com<mailto:csm9105@gmail.com>
> >> To: users@nifi.incubator.apache.org<mailto:
> users@nifi.incubator.apache.org>
> >>
> >> I am trying to push flowfile messages to ActiveMQ using a PutJMS
> >> processor. I am generating flow files using either a GetFile or
> >> GenerateFlowFile processor and feeding them to the PutJMS processor via
> >> a NIFI queue.
> >>
> >> Issue is that the PutJMS processor never keeps up with the queue and
> >> does not push messages out any quicker then at a rate of 30 seconds. I
> >> have the PutJMS setup as timer driven with Run Schedule of 0 secs.
> >>
> >> Any suggestions.
> >>
> >> Thanks,
> >>
> >> Chris
> >>
> >>
> >>
> >> --
> >> Chris Mangold
> >>
> >
> >
> >
> >
> > --
> > Chris Mangold
> > 301-471-5758 (c)
> > 301-898-7979 (h)
>
>



-- 
Chris Mangold
301-471-5758 (c)
301-898-7979 (h)

Mime
View raw message