nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Geercken" <uwe.geerc...@web.de>
Subject Aw: Re: Interest in a time series simulator processor for NiFi
Date Sat, 08 Jul 2017 11:30:42 GMT
Just my two cents...

I have created a processor that generates CSV data.
 
You can see an example here: http://datamelt.weebly.com/blog/nifi-processor-generatedata
 
It generates data based on word lists, regular expressions or purely random. Date data can
be generated in a way that the columns generated make sense. So e.g. a real date 2017-07-08
is generated randomly and you can have columns for year, month, day, weekday, week of year,
etc that corespond to this date.
 
And when you use word lists, it will generate data for catagories such as airlines, seasons,
car manufacturers, etc.
 
Hope this helps,
 
Uwe
 

Gesendet: Freitag, 07. Juli 2017 um 17:21 Uhr
Von: "Lee Laim (leelaim)" <leelaim@micron.com>
An: "dev@nifi.apache.org" <dev@nifi.apache.org>
Betreff: Re: Interest in a time series simulator processor for NiFi
Yes. It will be very useful in testing flows that will run in non-perfect environments. This
will be much more elegant than using several generate flow files in parallel with different
settings.

Thanks!
Lee

leelaim@micron.com



> On Jul 7, 2017, at 8:44 AM, Matt Burgess <mattyb149@gmail.com> wrote:
>
> Chris,
>
> This sounds great! IMO Realistic data generation in all forms is a great addition, looking
forward to your contribution!
>
> Regards,
> Matt
>
>
>> On Jul 7, 2017, at 10:18 AM, Chris Herrera <chris.herrera07@gmail.com> wrote:
>>
>> Hi All,
>>
>> I am trying to gauge interest in a processor I have written that generates realistic
time series data. I used the excellent GenerateFlowFile processor for a long time for load
testing, etc..., however, I needed something that mirrored more the semantics of a sensor,
and more importantly generated data like one. I have wrapped another Apache 2 licensed project
TSimulus[1] that generates realistic time series data driven by a JSON configuration file.
The idea is that this can be used to simulate a more IoT-like scenario. More than anything
I am just trying to see if this is something that has value outside of my little world.
>>
>> Regards,
>> Chris
>>
>> [1] - https://github.com/cetic/TSimulus[https://github.com/cetic/TSimulus]

Mime
View raw message