nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Payne <marka...@hotmail.com>
Subject Re: Cannot convert to Record a valid Avro schema
Date Tue, 27 Feb 2018 14:24:53 GMT
Juan,

OK, thanks. I was trying to understand what the intent was with that schema. So to make sure
that everyone
is on the same page:

If I use the following field in the schema:


"type": {"type": "array", "items": "int" }, "default": [ 0 ]

Then it works properly, giving us the default value of a 1-element array with the value 0
as the only element.

But if I use:


"type": {"type": "array", "items": "int" }, "default": 0

Then currently it throws an Exception because the default value is not an array. But what
you expect to happen
is for this case to be treated the same as if the schema had said:


"type": {"type": "array", "items": "int" }, "default": []

So that if the field is not specified, you get an empty array for the value.

Is that accurate?

Thanks
-Mark


On Feb 27, 2018, at 9:18 AM, Juan Pablo Gardella <gardellajuanpablo@gmail.com<mailto:gardellajuanpablo@gmail.com>>
wrote:

Thanks Mike, agree but Avro does not complain and it allows using it. The schema is used in
production and I cannot change it for now.

On Tue, 27 Feb 2018 at 11:17 Mike Thomsen <mikerthomsen@gmail.com<mailto:mikerthomsen@gmail.com>>
wrote:
That doesn't look like the right way to specify an empty array. This SO example fits about
what I'd expect:

https://stackoverflow.com/a/42140165/284538

So it should be default:[0]

On Tue, Feb 27, 2018 at 8:56 AM, Mark Payne <markap14@hotmail.com<mailto:markap14@hotmail.com>>
wrote:
Juan,

So the scenario that you laid out in the NIFI-4893 is not one that I've personally
encountered. What does it mean exactly to have an Avro schema with an "array" type
that has a value? In the example that you laid out, the field has:


"type": {"type": "array", "items": "int" }, "default": 0

In this case, what should be the value of this field if it's not specified? A single-element
array with the value of 0?
>From looking at the PR that was submitted, it appears to set the defaultValue to a new
(empty) ArrayList.
I would think that maybe it should set defaultValue to

new Integer[] {0};

But I am not certain of the semantics here.

Thanks
-Mark


On Feb 27, 2018, at 7:30 AM, Juan Pablo Gardella <gardellajuanpablo@gmail.com<mailto:gardellajuanpablo@gmail.com>>
wrote:

Hello team,

I could not fix the issue. I did a patch but it's solves the issue, but I believe is not the
correct solution. Anyone that knows Record Framework can help me on that?

Thanks in advance,
Juan

On Mon, 19 Feb 2018 at 22:33 Juan Pablo Gardella <gardellajuanpablo@gmail.com<mailto:gardellajuanpablo@gmail.com>>
wrote:
I saw an issue in a test :(. I will continue looking into current approach.

On Mon, 19 Feb 2018 at 22:23 Juan Pablo Gardella <gardellajuanpablo@gmail.com<mailto:gardellajuanpablo@gmail.com>>
wrote:
Hello team,

I filed an issue at https://issues.apache.org/jira/browse/NIFI-4893. I discovered using a
complex Avro schema. I've isolated the issue and also did a patch. At least, this solve the
issue but actually I don't know well the implications on that solution.

Please let me know what do you think. I have another issue related to Avro and Record, I will
file the issue tomorrow.

Thanks,
Juan




Mime
View raw message