nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koji Kawamura <ijokaruma...@gmail.com>
Subject Re: PutElasticsearchHttp error behaviour
Date Mon, 07 Nov 2016 05:09:58 GMT
Hello Gaspar,

I looked at the PutElasticsearchHttp code, it distinguishes following
two types of error:

1. Error occurs when the processor tries to connect Elasticsearch, but
it couldn't. E.g. the specified Elasticsearch URL is not correct and
java.net.ConnectException is thrown.
When this happens, the processor throws ProcessException, and NiFi
framework keeps the FlowFile in the queue, so that it can be retried
later.
This indicates that DataFlowManager(user) has to update the processor
configuration to make it work properly.

2. The processor successfully sends a request to Elasticsearch, but
Elasticsearch returns error response. This happens with operations
such as updating with a document Id that doesn't exist in the target
index.
When this happens, the processor routes the incoming FlowFile which
caused the error, to failure relationship.

I think the error you encountered the former type of error above (#1).
If the error was returned from Elasticsearch, it should be routed to
failure relationship as you suspected. Would you please let us know
about the specific error you got? A StackTrace would be helpful.

BTW, PutElasticsearchHttp also has REL_RETRY relationship, but it
seems it's not used currently.

Thanks!
Koji

On Thu, Nov 3, 2016 at 5:47 AM, Gaspar, Carson <gaspar@deshaw.com> wrote:
> When PutElasticsearchHttp gets an error from Elasticsearch, it _appears_ to
> never dequeue the problem flow file and just try it over and over again. It
> certainly doesn’t route the problem flow file to the retry or failure
> queues. This seems wrong to me – am I missing something?
>
> Nifi is from the Hortonworks nifi_2_0_0_0_579-1.0.0.2.0.0.0-579.el6.x86_64
> RPM.
>
>

Mime
View raw message