nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toivo Adams <>
Subject Re: Common data exchange formats and tabular data
Date Sun, 08 Nov 2015 14:54:22 GMT

Good overview.

>Avro is similar, but the schema must always be provided with the data. In 
>the case of NiFi DataFlows, it's likely more efficient to send the schema 
>once as an initialization packet (I can't remember the real term in NiFi), 
>then the rows can be streamed individually, in batches of user-defined
>sampled, etc. 

Do you mean "Initial Information Packet" or "IIP" ?
Mr. Morrison classical FBP includes such functionality, used often for

As far I know NiFi don't have such concept.

But NiFi ExecuteSQL uses Avro with schema for query result.
Result is one big FlowFile which includes both schema and all rows.
Processor just creates schema from JDBC metadata, writes it to Avro
and next all rows are written to Avro container.
Writing and reading such file is done using streaming, so result can very


View this message in context:
Sent from the Apache NiFi Developer List mailing list archive at

View raw message