nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toivo Adams <>
Subject Re: Common data exchange formats and tabular data
Date Sun, 08 Nov 2015 15:40:30 GMT

Benjamin has already done a lot of good work and it would very helpful we
can agree how to move on.

My first post was naive, there are much more things to consider.

It is probably impossible to select only one “correct data exchange format”
what all processors should use.

But can we agree one or two preferred data format what SQL and NoSQL
processors should support.
And all other other formats are supported using converter processors.

I my opinion preferred data exchange format should:

1. Support schema in the way or another.

2. Support streaming.

3. Support different data types (String, numeric types, date/time, binary)

4. Serialization/deserialization should be fast and efficient.

5. Widely used and has strong supporters.

6. Can be used in transformations, filtering, join, split, etc.

7. Can be converted to/and from other formats relatively easily.

Nice to have:

1. Nested data structures. For example Orders can contain order rows.

Or maybe we recommend all SQL and NoSQL processors should support two or
more input/output formats and user can select format using configuration?
Or separate sets of processors for different formats?


View this message in context:
Sent from the Apache NiFi Developer List mailing list archive at

View raw message