nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Burgess <mattyb...@apache.org>
Subject Re: ConvertRecord
Date Thu, 08 Feb 2018 20:58:13 GMT
Austin,

What version of NiFi are you using? I'm wondering if you're running
into [1] (fixed in 1.3.0), or [2] (fixed in 1.4.0), or something else.
You may want to change the types to be "optional", meaning a union
between null and the intended type. So for "PracticeId", try:

{"name": "PracticeId", "type": ["null", "string"], "default": null},

You may find that you don't need the "default": null in there once you
use the union type, and perhaps that can get you through this issue.

Another data point for debugging: before ConvertRecord, try
ExtractAvroMetadata and include "avro.schema" in the list to retrieve.
This will get the embedded schema into the avro.schema attribute, and
you can then compare it against what you have as the output schema.

Regards,
Matt

[1] https://issues.apache.org/jira/browse/NIFI-3879
[2] https://issues.apache.org/jira/browse/NIFI-4030

On Thu, Feb 8, 2018 at 3:50 PM, Austin Duncan <aduncan@pyaanalytics.com> wrote:
> All,
> I am trying to convert an avro record taken from a QueryDatabaseRecord
> processor and convert it into a json. I am trying to do this with the
> convert record processor using the avroreader and jsonrecordsetwriter
> controller services. My avro schema is being inferred by the avroreader and
> my json wrecordsetwriter is using the schema at the bottom of the message.
> Whenever I run my avro flowfiles through the processor i get a cannot parse
> data error on all files. I am not quite sure why and would like any
> information on the subject. I usually convert the avro to a json and then
> use jolttransforms to get the format but I think this new method might be
> better than jolt. Any help would be much appreciated.
> {
>  "type": "record",
>  "name": "Evoq",
> "namespace" : "my.example",
>  "fields": [{"name": "Practice",
>      "type":{
>         "type":"record",
>                 "name":"Practice",
>         "fields" : [
>             {"name": "PracticeId", "type": "string", "default": null},
>               {"name": "PracticeName", "type": "string", "default":
> null}]}},
>   {"name": "ScheduledPhysician",
>      "type":{
>         "type":"record",
>                 "name":"ScheduledPhysician",
>         "fields" : [
>             {"name": "PhysicianId", "type": "string", "default": null},
>               {"name": "PhysicianLastName", "type": "string", "default":
> null},
>               {"name": "PhysicianFirstName", "type": "string", "default":
> null},
>             {"name": "NPI", "type": "string" ,"default": null}]}},
>   {"name": "AttendingPhysician",
>      "type":{
>         "type":"record",
>                 "name":"AttendingPhysician",
>         "fields" : [
>             {"name": "PhysicianId", "type": "string" ,"default": null},
>               {"name": "PhysicianLastName", "type": "string" ,"default":
> null},
>               {"name": "PhysicianFirstName", "type": "string" ,"default":
> null},
>             {"name": "NPI", "type": "string" ,"default": null}]}},
>  {"name": "ReferringPhysician",
>      "type":{
>         "type":"record",
>                 "name":"ReferringPhysician",
>         "fields" : [
>             {"name": "PhysicianId", "type": "string" ,"default": null},
>               {"name": "PhysicianLastName", "type": "string" ,"default":
> null},
>               {"name": "PhysicianFirstName", "type": "string" ,"default":
> null},
>             {"name": "NPI", "type": "string" ,"default": null}]}},
>   {"name": "Patient",
>      "type":{
>         "type":"record",
>                 "name":"Patient",
>         "fields" : [
>             {"name": "PatientId", "type": "string" ,"default": null},
>               {"name": "FirstName", "type": "string" ,"default": null},
>               {"name": "LastName", "type": "string" ,"default": null},
>             {"name": "DateOfBirth", "type": "string" ,"default": null},
>                         {"name": "Gender", "type": "string" ,"default":
> null},
>                         {"name": "Address",
>                                "type":{
>                         "type":"record",
>                                 "name":"Address",
>                         "fields" : [
>             {"name": "Address1", "type": "string" ,"default": null},
>               {"name": "Address2", "type": "string" ,"default": null},
>               {"name": "City", "type": "string" ,"default": null},
>             {"name": "State", "type": "string" ,"default": null},
>                         {"name": "Zip", "type": "string" ,"default":
> null}]}},
>  {"name": "SSN","type": "string"}]}},
>
>  {"name": "Insurance",
>          "type":{
>           "type":"array",
>           "items": {
>           "name" : "Insurance",
>                          "type":"record",
>                          "fields" : [
>                           {"name" : "Payer", "type":  "string", "default":
> null},
>                           {"name" : "PolicyNumber", "type":"string",
> "default": null}]}}},
>
>  {"name": "Appointment",
>      "type":{
>         "type":"record",
>                 "name":"Appointment",
>         "fields" : [
>             {"name": "AppointmentId", "type": "string" ,"default": null},
>               {"name": "AppointmentTime", "type": "string" ,"default":
> null},
>               {"name": "AppointmentType", "type": "string" ,"default":
> null},
>               {"name": "AppointmentDescription", "type": "string"
> ,"default": null},
>               {"name": "AppointmentStatus", "type": "string" ,"default":
> null},
>             {"name": "EncounterID", "type": "string" ,"default": null}]}}
>  ]
> }
>
> --
> Austin Duncan
> Developer
> PYA Analytics
> 2220 Sutherland Avenue
> Knoxville, TN 37919
> 423-260-4172

Mime
View raw message