nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Wing <jvw...@gmail.com>
Subject Re: Best way to compare dates
Date Thu, 09 Jun 2016 16:48:28 GMT
Igor,

One way would be to format both dates as strings (like "20160609" in your
HDFS paths) first, then compare the two strings for equality.  In a
RouteOnAttribute expression:

${now():format("yyyyMMdd"):equals(${entryDate:format("yyyyMMdd")})}

If your goal is to merge the records into larger files, you might be able
to do this with MergeContent, if you have the Correlation Attribute set to
an attribute populated with the yyyyMMdd-formatteddate so it bins files by
day.

Thanks,

James

On Thu, Jun 9, 2016 at 7:37 AM, Igor Kravzov <igork.inexso@gmail.com> wrote:

> Hi,
>
> In my workflow data is coming form Kafka Topic in JSON format and saved ot
> HDFS directory by day.
>
> HDFS directory path constructed as /mypath/${now():format("yyyyMMdd")}
> JSON file contains attribute {... "adddate":"2016-06-07 17:06:17" }
>
> I want to avoid over-spill of a previous date files into "today's"
> directory.
> The thing is if add date is the same as taday's date, a want file merged
> to a bigger file and save to HDFS. Previous file not merged and saved as is.
>
> I am thinking to use EvaluateJsonPath to grab add date and to use
> RouteOnAttribute to direct to appropriate  path.
>
> How comparison would look like?
>
> Thanks in advance.
>

Mime
View raw message