nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marc Parisi <phroc...@apache.org>
Subject Re: NiFi for IoT Analytics
Date Mon, 01 Oct 2018 17:25:37 GMT
Julian,

Very good question. One of the issues I've been tackling has been 'how to
ease development' within MiNiFi C++. Users and developers will likely have
similar questions to yours, a common one I've heard on forums is "[h]ow can
I monitor for pull-down failure in this circuit?" There are a variety of
ways we could answer that and a variety of ways we could get that
information, but it's perfectly reasonable that consumers want this answer
to be sent to NiFi.

  One thing I've wrestled with is the fact that any given library you have
to support sensor or device management is going to fit every use case [1]
-- so how can we ease the analysis of that type of information without
creating individual stovepipes of flows. I've been thinking a lot about
enabling those who wish to use MiNIFi as a library, supporting a variety of
sensor/control libraries in a variety of a languages using MiNIFi's C
library as an interconnect. This might help you capture that bit and
monitor how long it's been there, facilitating the endpoints around it to
store that information. This piggybacks on what Joe said, but I think
enabling the python scripts, c programs, etc that already exist might help
improve the development process while also giving the capabilities that
people need.

This brings me to some other ideas we've been working on -- create useful
building blocks to facilitate this type of development. This may mean
providing facilities for the languages to keep and compute state -- as
consumables for these specific purposes. A great example might be to
monitor a sensor in python, capture and store that information with a few
library functions to send back through NiFi where you may perform further
evaluation/analysis on all sensor devices.

I know I haven't answered your question, but I'm trying to selfishly gauge
interest with this type of work. Some of it already exists, but knowing the
value of these capabilities is supremely helpful.

In the end is the state management of most importance to your evaluation or
am I completely off base?

  [1]
https://github.com/apache/nifi-minifi-cpp/tree/master/extensions/sensors --
specific to certain sensors and servos and likely not wildly helpful to
others.

  Thanks,
  Marc

On Sun, Sep 30, 2018 at 7:58 AM Julian Feinauer <
j.feinauer@pragmaticminds.de> wrote:

> Hi Nifi-team,
>
>
>
> I’m from the incubating plc4x project [1] and I am looking for a framework
> which is suitable for the management of IoT Datastreams and do some edge
> computing.
>
> As nifi is often times mentioned in relation with IoT I tried to find out
> what nifi realy does and how it would fit with our ideas (and also the
> MiNiFi Project seems to fit into this).
>
>
>
> From what I understood from the Docs and some Videos NiFi looks for me a
> bit like Apache Camel [2] as it is able to (dynamically) integrate
> different systems and manage the dataflow between them. So what I did not
> get exactly I how the payloads are managed between these Endpoints and how
> much of processing Nifi does itself and how much it delegates to other
> components (like e.g. Service Activater in EIP).
>
>
>
> What I am looking for is a framework which does some analysis of data
> streams coming from controllers that, e.g., control machines or robots.
> chrisdutz already prepared the first version of an NiFi Endpoint in th
> Plc4x Repo so we are already able to stream these datasets to NiFi. Whats
> unclear to me is how we could tackle some of the questions like “how long
> was this bit set” or “notify me when this signal is below a certain
> threshold for more than 30s” or so.
>
> Is this in the scope of NiFi or is NiFi more of an integration / data-flow
> layer which is absolutely agnostic of these processing blocks?
>
>
>
> I hope my questions are not too dumb or I’m not missing NiFis core too
> much with my current knowledge.
>
> I would be happy for some answers or some ideas about how to approach the
> questions stated above by some experienced users.
>
>
>
> Best
>
> Julian
>
>
>
> [1] http://plc4x.incubator.apache.org/
>
> [2] https://camel.apache.org/
>
> [3]
> https://github.com/apache/incubator-plc4x/tree/master/integrations/apache-nifi
>

Mime
View raw message