nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jorge Machado <jom...@me.com>
Subject Re: FlattenJson
Date Tue, 20 Mar 2018 13:50:52 GMT
We are using nifi for Workflow and we get from a database like job_status and job_name and
some nested json columns.  (30 columns)
We need to put it as attributes from the Flow file and not the content. For the first part
(columns without a json is done by groovy script) but then would be nice to use this standard
processor and instead of writing this to a flow content write it to attributes. 


Jorge Machado





> On 20 Mar 2018, at 14:47, Bryan Bende <bbende@gmail.com> wrote:
> 
> What would be the main use case for wanting all the flattened values
> in attributes?
> 
> If the reason was to keep the original content, we could probably just
> added an original relationship.
> 
> Also, I think FlattenJson supports flattening a flow file where the
> root is an array of JSON documents (although I'm not totally sure), so
> you'd have to consider what to do in that case.
> 
> On Tue, Mar 20, 2018 at 5:26 AM, Pierre Villard
> <pierre.villard.fr@gmail.com> wrote:
>> No I do see how this could be convenient in some cases. My comment was
>> more: you can certainly submit a PR for that feature, but it'll need to be
>> clearly documented using the appropriate annotations, documentation, and
>> property descriptions.
>> 
>> 2018-03-20 10:20 GMT+01:00 Jorge Machado <jomach@me.com>:
>> 
>>> Hi Pierre, I’m aware of that. So This means the change would not be
>>> accepted correct ?
>>> 
>>> Regards
>>> 
>>> Jorge Machado
>>> 
>>> 
>>> 
>>> 
>>> 
>>>> On 20 Mar 2018, at 09:54, Pierre Villard <pierre.villard.fr@gmail.com>
>>> wrote:
>>>> 
>>>> Hi Jorge,
>>>> 
>>>> I think this should be carefully documented to remind users that the
>>>> attributes are in memory. Doing what you propose would mean having in
>>>> memory the full content of the flow file as long as the flow file is
>>>> processed in the workflow (unless you remove attributes using
>>>> UpdateAttributes).
>>>> 
>>>> Pierre
>>>> 
>>>> 2018-03-20 7:55 GMT+01:00 Jorge Machado <jomach@me.com>:
>>>> 
>>>>> Hey guys,
>>>>> 
>>>>> I would like to change the FlattenJson Procerssor to be possible to
>>>>> Flatten to the attributes instead of Only to content. Is this a good
>>> Idea ?
>>>>> would the PR be accepted ?
>>>>> 
>>>>> Cheers
>>>>> 
>>>>> Jorge Machado
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>> 
>>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message