This sounds like just what I need. Thank you very much Matt. I'll dig in and give this a try. Thanks again to each of you guys who responded.  Cheers, Jim

On Fri, Aug 17, 2018 at 4:24 PM, Matt Burgess <> wrote:

You can use UpdateRecord for this, your input schema would have "last"
and "first" in it (and I think you can have an optional "myKey" field
so you can use the same schema for the writer), and the output schema
would have all three fields in it. Then you'd set the Replacement
Value Strategy to "Literal Value" and add a user-defined property in
UpdateRecord called "/myKey" set to "${myKey}". This will take the
value from the attribute myKey and put it at the root of each record
in a field called myKey.  Since this is JSON, you could do the same
with JoltTransformJSON, with a Default spec setting "myKey":
"${myKey}". Not sure which is faster in this case, since there appears
to be a single record.

This also works if there are multiple records in the flow file, as
long as the myKey field is to have the same value for all records
(since there is only one myKey attribute value for the whole flow
file).  If there are multiple records and they each need, you have a
"lookup" use case on your hands, where you'd want to match some value
against some lookup service, and it would fill in that field from the
value supplied by the lookup service (you'd use LookupService for
this). Or if all else fails, there is the Split pattern if you truly
do want/need to process one JSON object at a time.


On Fri, Aug 17, 2018 at 4:06 PM James McMahon <> wrote:
> I do appreciate your point, Tim and Lee. What if I do this instead: append select attributes to my data payload. Would that minimize the impact on RAM? Can I do that?
> More specifically, my data payload is a string representation of a JSON object, like so:
> {"last":"manson","first":"marilyn"}
> and I have an attribute named myKey that contains the value "123abc"
> Is there a processor that allows me to wind up with this string representation of JSON:
> {"last":"manson","first":"marilyn", "myKey":"123abc"}
> If I could do that, I could avoid loading the entire data payload into an attribute, and manipulate them in a python script called by ExecuteScript. I know how to do that, I don't know how to do the above with native processors.
> Thanks in advance for your help.
> On Fri, Aug 17, 2018 at 2:02 PM, Lee Laim <> wrote:
>> Jim,
>> I think the ExtractText processor with a large enough MaxCaptureGroup length (default :1024) will do that.      Though, I share Tim’s concerns when you scale up
>> Thanks,
>> Lee
>> > On Aug 17, 2018, at 11:52 AM, Timothy Tschampel <> wrote:
>> >
>> >
>> > This may not be applicable to your use case depending on message volume / # of attributes; but I would avoid putting payloads into attributes for scalability reasons (especially RAM usage).
>> >
>> >
>> >> On Aug 17, 2018, at 10:47 AM, James McMahon <> wrote:
>> >>
>> >> I have flowfiles with data payloads that represent small strings of text (messages consumed from AMQP queues). I want to create an attribute that holds the entire payload for downstream use. How can I capture the entire data payload of a flowfile in a new attribute on the flowfile? Thank you in advance for your help. -Jim
>> >