nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pierre Villard <pierre.villard...@gmail.com>
Subject Re: PutElasticsearchHttp error behaviour
Date Mon, 07 Nov 2016 18:15:23 GMT
Hi Gaspar,

I believe this is because 4294967295 is a too big value that cannot be
contained in an integer. I am not a ES expert but you should look at your
applicationId type if you can change it.

Hope this helps.

2016-11-07 19:09 GMT+01:00 Gaspar, Carson <gaspar@deshaw.com>:

> It stays in the queue on a field type error, e.g.:
> [2016-11-07 12:37:17,263][DEBUG][action.bulk              ] [Cybele]
> [estreamer_index_2016.11.07][0] failed to execute bulk item (index) index
> {[estreamer_index_2016.11.07][estreamer_doc][AVg_3ZmnIxO_esWdRpuM], source
> ...data...
> MapperParsingException[failed to parse [applicationId]]; nested:
> NumberFormatException[For input string: "4294967295"];
>         at org.elasticsearch.index.mapper.FieldMapper.parse(
> FieldMapper.java:329)
>         at org.elasticsearch.index.mapper.DocumentParser.
> parseObjectOrField(DocumentParser.java:311)
>         at org.elasticsearch.index.mapper.DocumentParser.
> parseValue(DocumentParser.java:438)
>         at org.elasticsearch.index.mapper.DocumentParser.
> parseObject(DocumentParser.java:264)
>         at org.elasticsearch.index.mapper.DocumentParser.
> parseDocument(DocumentParser.java:124)
>         at org.elasticsearch.index.mapper.DocumentMapper.parse(
> DocumentMapper.java:309)
>         at org.elasticsearch.index.shard.IndexShard.prepareCreate(
> IndexShard.java:529)
>         at org.elasticsearch.index.shard.IndexShard.
> prepareCreateOnPrimary(IndexShard.java:506)
>         at org.elasticsearch.action.index.TransportIndexAction.
> prepareIndexOperationOnPrimary(TransportIndexAction.java:214)
>         at org.elasticsearch.action.index.TransportIndexAction.
> executeIndexRequestOnPrimary(TransportIndexAction.java:223)
>         at org.elasticsearch.action.bulk.TransportShardBulkAction.
> shardIndexOperation(TransportShardBulkAction.java:327)
>         at org.elasticsearch.action.bulk.TransportShardBulkAction.
> shardOperationOnPrimary(TransportShardBulkAction.java:120)
>         at org.elasticsearch.action.bulk.TransportShardBulkAction.
> shardOperationOnPrimary(TransportShardBulkAction.java:68)
>         at org.elasticsearch.action.support.replication.
> TransportReplicationAction$PrimaryPhase.doRun(TransportReplicationAction.
> java:648)
>         at org.elasticsearch.common.util.concurrent.AbstractRunnable.
> run(AbstractRunnable.java:37)
>         at org.elasticsearch.action.support.replication.
> TransportReplicationAction$PrimaryOperationTransportHandl
> er.messageReceived(TransportReplicationAction.java:279)
>         at org.elasticsearch.action.support.replication.
> TransportReplicationAction$PrimaryOperationTransportHandl
> er.messageReceived(TransportReplicationAction.java:271)
>         at org.elasticsearch.transport.RequestHandlerRegistry.
> processMessageReceived(RequestHandlerRegistry.java:77)
>         at org.elasticsearch.transport.TransportService$4.doRun(
> TransportService.java:376)
>         at org.elasticsearch.common.util.concurrent.AbstractRunnable.
> run(AbstractRunnable.java:37)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
>         at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.NumberFormatException: For input string: "4294967295"
>         at java.lang.NumberFormatException.forInputString(
> NumberFormatException.java:65)
>         at java.lang.Integer.parseInt(Integer.java:583)
>         at java.lang.Integer.parseInt(Integer.java:615)
>         at org.elasticsearch.common.xcontent.support.
> AbstractXContentParser.intValue(AbstractXContentParser.java:127)
>         at org.elasticsearch.index.mapper.core.IntegerFieldMapper.
> innerParseCreateField(IntegerFieldMapper.java:293)
>         at org.elasticsearch.index.mapper.core.NumberFieldMapper.
> parseCreateField(NumberFieldMapper.java:241)
>         at org.elasticsearch.index.mapper.FieldMapper.parse(
> FieldMapper.java:321)
>         ... 22 more
>
> -----Original Message-----
> From: Koji Kawamura [mailto:ijokarumawak@gmail.com]
> Sent: Monday, November 07, 2016 12:10 AM
> To: users@nifi.apache.org
> Subject: Re: PutElasticsearchHttp error behaviour
>
> Hello Gaspar,
>
> I looked at the PutElasticsearchHttp code, it distinguishes following two
> types of error:
>
> 1. Error occurs when the processor tries to connect Elasticsearch, but it
> couldn't. E.g. the specified Elasticsearch URL is not correct and
> java.net.ConnectException is thrown.
> When this happens, the processor throws ProcessException, and NiFi
> framework keeps the FlowFile in the queue, so that it can be retried later.
> This indicates that DataFlowManager(user) has to update the processor
> configuration to make it work properly.
>
> 2. The processor successfully sends a request to Elasticsearch, but
> Elasticsearch returns error response. This happens with operations such as
> updating with a document Id that doesn't exist in the target index.
> When this happens, the processor routes the incoming FlowFile which caused
> the error, to failure relationship.
>
> I think the error you encountered the former type of error above (#1).
> If the error was returned from Elasticsearch, it should be routed to
> failure relationship as you suspected. Would you please let us know about
> the specific error you got? A StackTrace would be helpful.
>
> BTW, PutElasticsearchHttp also has REL_RETRY relationship, but it seems
> it's not used currently.
>
> Thanks!
> Koji
>
> On Thu, Nov 3, 2016 at 5:47 AM, Gaspar, Carson <gaspar@deshaw.com> wrote:
> > When PutElasticsearchHttp gets an error from Elasticsearch, it
> > _appears_ to never dequeue the problem flow file and just try it over
> > and over again. It certainly doesn’t route the problem flow file to
> > the retry or failure queues. This seems wrong to me – am I missing
> something?
> >
> > Nifi is from the Hortonworks
> > nifi_2_0_0_0_579-1.0.0.2.0.0.0-579.el6.x86_64
> > RPM.
> >
> >
>

Mime
View raw message