nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "BOUKRAA, Kacem" <bk_bouk...@esi.dz>
Subject Re: PutKafka taks number is huge
Date Mon, 15 Feb 2016 19:16:58 GMT
Jeff thanks,
It seems like the time interval was the reason for this extra load as you
mentioned .
Thanks for your help. You're no longer a begginer x)

Sent from my Nexus7
On Feb 15, 2016 11:36 AM, "Jeff - Data Bean Australia" <
databean.au@gmail.com> wrote:

> Hi Kacem,
>
> I am also new to NiFi. Just try to put some thoughts and see whether it
> makes sense to you.
>
> I can't say it is normal, but I may understand why the task number was so
> high:
>
>
>    - PutKafka is annotated as @TriggerWhenEmpty. With this, PutKafka will
>    be triggered as often as the SchedulingAgent want to. The scheduling agent
>    in your case was Timer Driven Scheduling Agent I think. And by default, the
>    value of Run schedule is 0 sec, which means Timer Driven Scheduling Agent
>    will schedule PutKafka without waiting.
>    - When PutKafka tries to poll data/FlowFile out of a ProcessSession, I
>    guess it doesn't block when FlowFile is not ready.
>
>
> So I guess this is the reason the number of tasks for PutKafka is so high.
> And I suspect it consumed a lot of CPU time also. Maybe you can put a small
> value, such as 1 ms to Run Schedule to slow it down if it makes sense to
> you.
>
> Thanks,
> Jeff
>
> On Mon, Feb 15, 2016 at 10:07 PM, BOUKRAA, Kacem <bk_boukraa@esi.dz>
> wrote:
>
>> Hello Nifi users community,
>>
>> ​So i'm using a ListenHTTP processor followed by PutKafka,
>> I've created the Kafka topic. launched the producer, but the number of
>> task is very high, is that normal?
>>
>> My config:
>>
>>
>> ​Thanks
>> ​​
>>
>> --
>>
>> [image: Image]
>> *Kacem BOUKRAA*
>>
>>
>>
>
>
> --
> Data Bean - A Big Data Solution Provider in Australia.
>

Mime
View raw message