nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Thomsen <mikerthom...@gmail.com>
Subject Re: PutDatabaseRecord won't put Timestamp field
Date Thu, 05 Apr 2018 19:56:07 GMT
1.5

Thanks,

Mike

On Thu, Apr 5, 2018 at 3:40 PM, Matt Burgess <mattyb149@apache.org> wrote:

> Mike,
>
> I can't reproduce this, I use the same DDL and Avro schema, with data
> coming in from the SiteToSiteProvenanceReportingTask and going to
> Postgres, and it works fine. What version of NiFi are you using?
>
> Regards,
> Matt
>
>
> On Thu, Apr 5, 2018 at 3:05 PM, Mike Thomsen <mikerthomsen@gmail.com>
> wrote:
> > Found these errors in the Docker logs:
> >
> > postgres_1       | 2018-04-05 18:33:22.183 UTC [51] ERROR:  column
> > "timestamp_field" is of type timestamp without time zone but expression
> is
> > of type bigint at character 282
> > postgres_1       | 2018-04-05 18:33:22.183 UTC [51] HINT:  You will need
> to
> > rewrite or cast the expression.
> > postgres_1       | 2018-04-05 18:33:22.183 UTC [51] STATEMENT:  INSERT
> INTO
> > provenance (componentid, componentname, componenttype, details, entityid,
> > entitysize, entitytype, eventid, eventtype, processgroupid,
> > processgroupname, record_count, schema_name, timestamp_field) VALUES
> > ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12,$13,$14)
> > nifi2            | 2018-04-05 18:33:22,184 WARN [Timer-Driven Process
> > Thread-2] o.a.n.p.standard.PutDatabaseRecord
> > PutDatabaseRecord[id=dfdd3dd1-21ee-16e2-09cc-159b7c7f8f54] Failed to
> process
> > StandardFlowFileRecord[uuid=8cf3b521-ac0b-4149-a60e-5fa7b2d2b3c5,claim=
> StandardContentClaim
> > [resourceClaim=StandardResourceClaim[id=1522953007606-749,
> > container=default, section=749], offset=434600,
> > length=528],offset=0,name=4306158395541,size=528] due to
> > java.sql.BatchUpdateException: Batch entry 0 INSERT INTO provenance
> > (componentid, componentname, componenttype, details, entityid,
> entitysize,
> > entitytype, eventid, eventtype, processgroupid, processgroupname,
> > record_count, schema_name, timestamp_field) VALUES
> > ('8d08a7d3-0162-1000-7216-7e0b426e774a','EvaluateJsonPath','
> EvaluateJsonPath',NULL,'26a1efa3-bcd3-4015-8a43-2a2c43c05714',66307,'org.
> apache.nifi.flowfile.FlowFile','e841aa4e-a7d3-4a6e-b78c-
> 4e785f53b60e','ATTRIBUTES_MODIFIED','8c5c89d5-0162-1000-
> 4395-a38d1c7b0b2f','Mongo
> > ES Test',NULL,NULL,1522952968586) was aborted: ERROR: column
> > "timestamp_field" is of type timestamp without time zone but expression
> is
> > of type bigint
> >
> > I have the following DDL and Avro schema:
> >
> > create table provenance (
> > id serial,
> > componentId varchar(128),
> > componentName varchar(256),
> > componentType varchar(128),
> > details varchar(256),
> > entityId varchar(256),
> > entitySize int,
> > entityType varchar(128),
> > eventId varchar(128),
> > eventType varchar(128),
> > processGroupId varchar(128),
> > processGroupName varchar(128), record_count int,
> > schema_name varchar(64),
> > timestamp_field timestamp
> > )
> >
> > {
> > "type": "record",
> > "name": "ProvenanceEvent",
> > "fields": [
> > { "name": "componentId", "type": ["null", "string"] },
> > { "name": "componentName", "type": ["null", "string"] },
> > { "name": "componentType", "type": ["null", "string"] },
> > { "name": "details", "type": ["null", "string"] },
> > { "name": "entityId", "type": ["null", "string"] },
> > { "name": "entitySize", "type": ["null", "int"] },
> > { "name": "entityType", "type": ["null", "string"] },
> > { "name": "eventId", "type": ["null", "string"] },
> > { "name": "eventType", "type": ["null", "string"] },
> > { "name": "processGroupId", "type": ["null", "string"] },
> > { "name": "processGroupName", "type": ["null", "string"] },
> > { "name": "record_count", "type": ["null", "int"] },
> > { "name": "schema_name", "type": ["null", "string"] },
> > { "name": "timestamp_field", "type": "long", "logicalType":
> > "timestamp-millis" }
> > ]
> > }
> >
> > I set the JsonTreeReader to use this formatting option for reading in the
> > original ISO8601 string:
> >
> > yyyy-MM-dd'T'HH:mm:ssZ
> >
> > Everything looks fine until it gets to the processor. Any ideas?
> >
> > Thanks,
> >
> > Mike
>

Mime
View raw message