nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Boris Tyukin <bo...@boristyukin.com>
Subject backpressure issue
Date Wed, 22 May 2019 18:23:09 GMT
Hi guys,

we run on 1.9.2 and experience an interesting issue with one of the custom
groovy processors.

It batches 2000 flowfiles (sessing.get(2000)) and routes some to a waiting
relationship for retries. that relationship has backpressure set to 5000
flowfiles.

What happens is we can see often 6000-8000 flowfiles there while according
to backpressure setting we should only see 5000 max.

Is this by design or flowfiles can overflow or something wrong with the way
our custom processor routes them?

code is really easy and based on some conditional logic it would either

let the flowfiles pass downstream with

session.transfer(flowFile, REL_SUCCESS)


or route flowfile to a waiting relationship like so

session.penalize(flowFile)
session.transfer(flowFile, REL_WAIT)

then session is commited for the entire batch
session.commit()

Mime
View raw message