nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mike Harding <mikeyhard...@gmail.com>
Subject Re: Is it possible to reference python requests module in ExecuteScript?
Date Sat, 06 May 2017 10:22:04 GMT
Thanks for all the suggestions. Regards using Groovy I did try and use it
to solve my problem but just couldn't get it to work correctly. I tried to
implement something similar to the following solution given here (
http://stackoverflow.com/questions/24827855/groovy-httpbuilder-issues-with-cookies)
to address a cookie authentication issue but the cookie didn't seem to
attach/work with follow-on requests and still gave me 401 but I managed to
get something working in python hence why I was trying to use that.

I'm wondering whether I should just use ExecuteProcess/ExecuteStreamCommand
as a quick fix?

Mike

On 6 May 2017 at 09:29, Giovanni Lanzani <giovannilanzani@godatadriven.com>
wrote:

> Please do not remove the Python scripting facilities.
>
> I believe most people experience it as very slow because (ok, Python is
> slow) they only get a flow file at a time.
>
> I think Matt pointed out in this ML once that you can use
>
> flowfiles = session.get(1000)
> for flowfile in filter(None, flowfiles):
>     # do things
>
> In that case Jython will be kept alive much longer. Or am I missing
> something?
>
> Giovanni
>
> > -----Original Message-----
> > From: Joe Witt [mailto:joe.witt@gmail.com]
> > Sent: Saturday, May 6, 2017 1:12 AM
> > To: users@nifi.apache.org
> > Subject: Re: Is it possible to reference python requests module in
> > ExecuteScript?
> >
> > It is worth discussing whether there is sufficient value to warrant
> keeping
> > jython/python support in the processors or whether we should pull it.
> It is
> > certainly something we can document as being highly limited but we don't
> > really know how limited.  Frankly given the performance I've seen with
> it I'd be
> > ok removing it entirely.  One is better off calling the script via a
> system call.
> > Groovy is one that I've seen perform very well and be fully featured.
> >
> > On Fri, May 5, 2017 at 6:38 PM, Russell Bateman <russ@windofkeltia.com>
> > wrote:
> > > We really want to use ExecuteScript because our end users are
> Pythonistas.
> > > They tend to punctuate their flows with the equivalent of PutFile and
> > > GetFile with Python scripts doing stuff on flowfiles that pass out of
> > > NiFi before returning into NiFi.
> > >
> > > However, we find it nearly impossible to replace even the tamest of
> > > undertakings. If there were a good set of NiFi/Python shims that, from
> > > PyCharm, etc., gave us the ability to prototype, test and debug before
> > > copying and pasting into ExecuteScript, that would be wonderful. It
> > > hasn't worked out that way. Most of our experience is copying, pasting
> > > into the processor property, only to find something wrong, sometimes
> > > syntax, sometimes something runtime.
> > >
> > > On their behalf, I played with this processor a few hours a while back.
> > > Another colleague too. Googling this underused tool hasn't been
> > > helpful, so the overall experience is negative so far. I can get most
> > > of the examples out there to work, but as soon as I try to do "real"
> > > work from my point of view, my plans sort of cave in.
> > >
> > > Likely the Groovy and/or Ruby options are better? But, we're not
> > > Groovy or Ruby guys here. I understand the problems with this tool and
> > > so I understand what the obstacles are to it growing stronger. The
> > > problems won't yield to a few hours one Saturday afternoon. Better
> > > problem-logging underneath and
> > > better- and more lenient Python support on top. The second one is
> > > tough, though.
> > >
> > > My approach is to minimize those black holes these guys put into their
> > > flows by creating custom processors for what I can't solve using
> > > standard processors.
> > >
> > > Trying not to be too negative here...
> > >
> > >
> > > On 05/05/2017 04:09 PM, Andre wrote:
> > >
> > > Mike,
> > >
> > > I believe it is possible to use requests under jython, however the
> > > process isn't very intuitive.
> > >
> > > I know one folk that if I recall correctly has used it. Happy to try
> > > to find out how it is done.
> > >
> > > Cheers
> > >
> > > On Sat, May 6, 2017 at 4:57 AM, Mike Harding <mikeyharding@gmail.com>
> > wrote:
> > >>
> > >> Hi All, I'm now looking at using ExecuteScript and python engine to
> > >> execute HTTP requests using the requests module. I've tried
> > >> referencing requests the module but when I try to import requests I
> > >> get a module reference error.
> > >> I downloaded the module from here >
> > >> https://pypi.python.org/pypi/requests
> > >> Not sure why it isnt picking it up. Ive tried referencing the
> > >> directory and the .py directly with no success.
> > >> Any ideas where im going wrong?
> > >> Cheers,
> > >> Mike
>

Mime
View raw message