nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: ValidateRecord1.4.0 vs ConvertJsonToAvro1.4.0 regarding required field in nested object
Date Wed, 06 Dec 2017 14:52:27 GMT
Hey Martin,

Thanks for flagging this and for providing the template! It makes it really easy to reproduce.
It looks like there is in fact a bug. Whenever the Schema is created from the AvroSchemaRegistry,
it is marking all fields except for the top-level fields as as 'nullable'.

I created a JIRA [1] to track the issue and should have a PR up soon to address it.

Many Thanks!
-Mark

[1] https://issues.apache.org/jira/browse/NIFI-4671


On Dec 6, 2017, at 5:25 AM, Martin Mucha <alfonz19@gmail.com<mailto:alfonz19@gmail.com>>
wrote:

I don't really understand what you're asking for...

In attachment you have nifi template,

avro template is:

{
  "name": "aRecord",
  "type": "record",
  "namespace": "a",
  "fields": [
    {
      "name": "a",
      "type": {
        "name": "bRecord",
        "type":"record",
        "fields": [
          { "name": "b", "type": "string"}
        ]
      }
    }

  ]
}


and incorrectly validated json file is:

{"a":{}}


In given flow it's validated as valid, although required filed b is missing. ConvertJsonToAvro
on the other hand rejects the very same json using the very same avro schema.

Is this all you need? If not, what do you need from me? I probably don't have 'reproducible
repository' -- I don't even know what that is.

Martin.

2017-12-06 11:07 GMT+01:00 Juan Pablo Gardella <gardellajuanpablo@gmail.com<mailto:gardellajuanpablo@gmail.com>>:

Could you share a reproducible repo or files?

El mié., 6 de dic. de 2017 07:00, Martin Mucha <alfonz19@gmail.com<mailto:alfonz19@gmail.com>>
escribió:
Hi,

I have JSON like:

{
  "a": {
    "b": "1"
  }
}

and corresponding avro schema (written for the sake of this e-mail, need not to be 100% accurate)

{
  "name": "aRecord",
  "type": "record",
  "namespace": "a",
  "fields": [
    {
      "name": "a",
      "type": {
        "name": "bRecord",
        "type":"record",
        "fields": [
          { "name": "b", "type": "string"}
        ]
      }
    }

  ]
}

In ConvertJsonToAvro processor, json missing field "b":

{"a":{}}

will be rejected, while in ValidateRecord it will be accepted as valid (which is not valid
according to schema). Is there anything I can do about it? Is it bug?

thanks,
Martin.

<fail.xml>

Mime
View raw message