nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@gmail.com>
Subject Re: Lua usage in ExecuteScript Processor
Date Wed, 04 May 2016 21:29:56 GMT
Mahdu,

The incoming flow file's content is raw bytes, so you'd either expect/require a particular
format and decode the bytes, or let the user choose (in ExecuteScript this can be with a dynamic
property), or try to parse the "mime.type" attribute to see if there is a character set defined.

Regards,
Matt


> On May 4, 2016, at 3:14 PM, Madhukar Thota <madhukar.thota@gmail.com> wrote:
> 
> Thanks Matt for the explanation.  I will try groovy but before i try what format of data
we get in inputstream( byte array)? i will try to see if i can decode in native lua.
> 
>> On Wed, May 4, 2016 at 1:21 PM, Matt Burgess <mattyb149@gmail.com> wrote:
>> Madhu,
>> 
>> Unfortunately, the LuaJ script engine resolves classes using the
>> system class loader as a parent class loader, rather than the current
>> thread's context class loader. This means LuaJ only has access to the
>> classes defined in JARs in the lib/ folder (not even lib/bootstrap).
>> The Module Directory property is useless for LuaJ at present, meaning
>> even if you add that JAR to the Module Directory property it still
>> won't work. Theoretically you'd add the JARs you want to the lib/
>> folder and restart NiFi, but then you're risking all sorts of bad news
>> and interactions.
>> 
>> The bottom line is that LuaJ should probably only be used to leverage
>> business logic written in Lua, not Java. If you want access to Java
>> libraries, I'd use another script engine such as Groovy.
>> 
>> Regards,
>> Matt
>> 
>> On Wed, May 4, 2016 at 11:39 AM, Madhukar Thota
>> <madhukar.thota@gmail.com> wrote:
>> > Hey Matt,
>> >
>> > Do you know how to call java classes in lua?
>> >
>> > i am trying to call java class org.apache.commons.io.IOUtils  like this:
>> >
>> > local io = luajava.bindClass("org.apache.commons.io.IOUtils")
>> >
>> > but nifi execurescript processor is complaning class not found.
>> >
>> > failed to process session due to org.luaj.vm2.LuaError: script:98 vm error:
>> > java.lang.ClassNotFoundException: org.apache.commons.io.IOUtils:
>> > org.luaj.vm2.LuaError: script:98 vm error: java.lang.ClassNotFoundException:
>> > org.apache.commons.io.IOUtils
>> >
>> >
>> >
>> > Any help here?
>> >
>> > On Thu, Apr 21, 2016 at 10:58 AM, Madhukar Thota <madhukar.thota@gmail.com>
>> > wrote:
>> >>
>> >> Made some progess on loading the lua files from modules directory. In my
>> >> case all my lua files and .so files are in modules diretory. I placed the
>> >> directory in nifi installation folder.
>> >>
>> >> Eg: lua_modules/common_log_format.lua
>> >>
>> >> in my script i am calling the luascript as follows:
>> >>
>> >> local clf = require 'lua_modules.common_log_format'
>> >>
>> >> It is reading the lua script without any issue, now the problem is
>> >> common_log_format.lua is dependent on lpeg module which is .so file (
>> >> lpeg.so). The question is, Can we read .so files in nifi luaj libarary.
If
>> >> so how can i parse .so files?
>> >>
>> >>
>> >> On Wed, Apr 20, 2016 at 5:21 PM, Madhukar Thota <madhukar.thota@gmail.com>
>> >> wrote:
>> >>>
>> >>> I am trying to read the lua file this way, but its not working. How
to
>> >>> read the lua files from module directory and use it in execution?
>> >>>
>> >>> luajava.LuaState = luajava.LuaStateFactory.newLuaState()
>> >>>
>> >>>
>> >>> luajava.LuaState.openLibs()
>> >>> luajava.LuaState.LdoFile("common_log_format.lua");
>> >>>
>> >>>
>> >>> On Wed, Apr 20, 2016 at 4:29 PM, Madhukar Thota
>> >>> <madhukar.thota@gmail.com> wrote:
>> >>>>
>> >>>> Thanks Matt. This will be helpful to get started. I will definitely
>> >>>> contribute back to community once i have working script. One more
question,
>> >>>> Can i call the lua modues in the script with require statement like
this
>> >>>> local lpeg = require "lpeg"?
>> >>>>
>> >>>> -Madhu
>> >>>>
>> >>>>
>> >>>>
>> >>>> On Wed, Apr 20, 2016 at 3:11 PM, Matt Burgess <mattyb149@gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>> Madhu,
>> >>>>>
>> >>>>> I know very little about Lua, so I haven't tried making a Lua
version
>> >>>>> of my JSON-to-JSON scripts/blogs (funnifi.blogspot.com), but
here's
>> >>>>> something that works to get you started. The following Luaj
script creates a
>> >>>>> flow file, writes to it, adds an attribute, then transfers it
to success.
>> >>>>> Hopefully you can use your Lua scripts inline by calling their
functions and
>> >>>>> such from the OutputStreamCallback proxy (the process method
below). If you
>> >>>>> get something working and would like to share, I would very
much appreciate
>> >>>>> it!
>> >>>>>
>> >>>>> local writecb =
>> >>>>> luajava.createProxy("org.apache.nifi.processor.io.OutputStreamCallback",
{
>> >>>>> process = function(outputStream)
>> >>>>> outputStream:write("This is flow file content from Lua")
>> >>>>> end
>> >>>>> })
>> >>>>> flowFile = session:create()
>> >>>>> flowFile = session:putAttribute(flowFile, "lua.attrib", "Hello
from
>> >>>>> Lua!")
>> >>>>> flowFile = session:write(flowFile, writecb)
>> >>>>> session:transfer(flowFile, REL_SUCCESS)
>> >>>>>
>> >>>>>
>> >>>>> Regards,
>> >>>>> Matt
>> >>>>>
>> >>>>> On Tue, Apr 19, 2016 at 1:15 PM, Madhukar Thota
>> >>>>> <madhukar.thota@gmail.com> wrote:
>> >>>>>>
>> >>>>>> Friends,
>> >>>>>>
>> >>>>>> Can anyone share an sample example on how to use Lua in
ExecuteScript
>> >>>>>> Processor? We have bunch of lua scripts which we would like
to use for data
>> >>>>>> processing.
>> >>>>>>
>> >>>>>> Any help is appreciated.
>> >>>>>>
>> >>>>>> Thanks
>> >>>>>> Madhu
>> >>>>>
>> >>>>>
>> >>>>
>> >>>
>> >>
>> >
> 

Mime
View raw message