nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tomislav Novosel <to.novo...@gmail.com>
Subject Re: Modify Flowfile attributes
Date Tue, 29 Jan 2019 15:06:28 GMT
Hi Matt, thanks for suggestions. But performance is not crucial here.
This is code i tried. but I get error: "AttributeError: 'NoneType' object
has no attribute 'getAttribute' at line number 4"
If I remove code from line 6 to line 14, it works with some default
attribute values for year_extracted and week_extracted, otherwise i get
error form above.

Tom

from datetime import datetime, timedelta, date

flowFile = session.get()
file_name = flowFile.getAttribute('filename')

date_file = file_name.split("_")[6]
date_final = date_file.split(".")[0]
date_obj = datetime.strptime(date_final,'%y%m%d')
date_year = date_obj.year
date_day = date_obj.day
date_month = date_obj.month

week = date(year=date_year, month=date_month, day=date_day).isocalendar()[1]
year = date(year=date_year, month=date_month, day=date_day).isocalendar()[0]

if (flowFile != None):
flowFile = session.putAttribute(flowFile, "year_extracted", year)
flowFile = session.putAttribute(flowFile, "week_extracted", week)
session.transfer(flowFile, REL_SUCCESS)
session.commit()

On Tue, 29 Jan 2019 at 15:53, Matt Burgess <mattyb149@apache.org> wrote:

> Tom,
>
> Keep in mind that you are using Jython not Python, which I mention
> only to point out that it is *much* slower than the native Java
> processors such as UpdateAttribute, and slower than other scripting
> engines such as Groovy or Javascript/Nashorn.
>
> If performance/throughput is not a concern and you're more comfortable
> with Jython, then Jerry's suggestion of session.putAttribute(flowFile,
> attributeName, attributeValue) should do the trick. Note that if you
> are adding more than a couple attributes, it's probably better to
> create a dictionary (eventually/actually, a Java Map<String,String>)
> of attribute name/value pairs, and use putAllAttributes(flowFile,
> attributes) instead, as it is more performant.
>
> Regards,
> Matt
>
> On Tue, Jan 29, 2019 at 9:25 AM Tomislav Novosel <to.novosel@gmail.com>
> wrote:
> >
> > Thanks for the answer.
> >
> > Yes I know I can handle that with Expression language and
> UpdateAttribute processor, but this is specific case on my work and I think
> Python
> > is better and more simple solution. I need to calc that with python
> script.
> >
> > Tom
> >
> > On Tue, 29 Jan 2019 at 15:18, John McGinn <amruginn-nifi@yahoo.com>
> wrote:
> >>
> >> Since you're script shows that "filename" is an attribute of your
> flowfile, you could use the UpdateAttribute processor.
> >>
> >> If you right click on UpdateAttribute and choose ShowUsage, then choose
> Expression Language Guide, it shows you the things you can handle.
> >>
> >> Something along the lines of ${filename:getDelimitedField(6,'_')}, if I
> understand the Groovy code correctly. I did a GenerateFlowFIle to an
> UpdateAttribute processor setting filename to "1_2_3_4_5_6.2_abc", then
> sent that to another UpdateAttribute with the getDelimitedField() I listed
> and I received 6.2. Then another UpdateAttribute could parse the 6.2 for
> the second substring, or you might be able to chain them in the existing
> UpdateProcessor.
> >>
> >>
> >> --------------------------------------------
> >> On Tue, 1/29/19, Tomislav Novosel <to.novosel@gmail.com> wrote:
> >>
> >>  Subject: Modify Flowfile attributes
> >>  To: users@nifi.apache.org
> >>  Date: Tuesday, January 29, 2019, 9:04 AM
> >>
> >>  Hi all,
> >>  I'm trying to calculate week number and date
> >>  from filename using ExecuteScript processor and Jython. Here
> >>  is python script.How can I add calculated
> >>  attributes week and year to flowfile?
> >>  Please help, thank you.Tom
> >>  P.S. Maybe I completely missed with this script.
> >>  Feel free to correct me.
> >>
> >>  import
> >>  jsonimport java.iofrom org.apache.commons.io import
> >>  IOUtilsfrom java.nio.charset import
> >>  StandardCharsetsfrom org.apache.nifi.processor.io import
> >>  StreamCallbackfrom datetime import datetime, timedelta, date
> >>  class PyStreamCallback(StreamCallback):
> >>  def __init__(self, flowfile):
> >>  self.ff = flowfile
> >>         pass
> >>  def process(self, inputStream, outputStream):
> >>  file_name =
> >>  self.ff.getAttribute("filename")
> >>  date_file =
> >>  file_name.split("_")[6]
> >>  date_final =
> >>  date_file.split(".")[0]
> >>  date_obj =
> >>  datetime.strptime(date_final,'%y%m%d')
> >>  date_year =
> >>  date_obj.year
> >>    date_day =
> >>  date_obj.day
> >>   date_month =
> >>  date_obj.month
> >>          week = date(year=date_year, month=date_month,
> day=date_day).isocalendar()[1]
> >>  year =
> >>  date(year=date_year, month=date_month, day=date_day).isocalendar()[0]
> >>  flowFile =
> >>  session.get()if (flowFile != None):
> >>  session.transfer(flowFile, REL_SUCCESS)
> >>  session.commit()
>

Mime
View raw message