nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Toivo Adams <toivo.ad...@gmail.com>
Subject Re: importing data from postgresql to disk using NIFI
Date Sat, 08 Oct 2016 12:53:31 GMT
Maybe we should write Other type object as Avro Byte sequence?This way
we preserve content.
For example Postgres getObject() should return PGpolygon or PGCircle
object (or any other PGObject subclass object).And next processor
could convert Avro bytes sequence back back to  PGObject object.
ThanksToivo


2016-10-05 11:11 GMT+03:00 Toivo Adams <toivo.adams@gmail.com>:

> Hi Selvam, Matt
>
>
>
> We can improve error reporting. Message
>
>      java.lang.IllegalArgumentException: createSchema: Unknown SQL type
> 1111 cannot be converted to Avro type
>
> should contain also column name (and possibly table name).
>
> Please, can you create Jira ticket?
>
>
>
> OTHER type
>
> Maybe we can try to use getObject() method and convert content to String
> using toString() method.
>
> Result might be nonsense, but what else we can do?
>
> We can catch any exceptions during getObject() and conversion and set Avro
> value to null.
>
>
>
> Thanks
>
> Toivo
>
> 2016-10-05 10:15 GMT+03:00 Selvam Raman <selmna@gmail.com>:
>
>> I found one table which has data type as Double precession and
>> geometry(postgresql).
>>
>> I hope we need to cast it to double and text. But not sure will it get
>> the data without corrupt.
>> On Oct 5, 2016 12:38 AM, "Matt Burgess" <mattyb149@gmail.com> wrote:
>>
>>> Selvam,
>>>
>>> Do you know which table is giving that error, and what columns are in
>>> it? Type 1111 is "other", which implies it contains non-standard values. It
>>> is possible we could improve ExecuteSQL to try and treat it like a string,
>>> but that wouldn't likely work for all cases, so it might be performing as
>>> expected for unknown types/values.
>>>
>>> Regards,
>>> Matt
>>>
>>>
>>> On Oct 4, 2016, at 6:30 PM, Selvam Raman <selmna@gmail.com> wrote:
>>>
>>> Hi,
>>>
>>> i am using listdatabasetable to list out the table then executing
>>> executesql to fetch the data and finally put file to save the data.
>>>
>>> ExecuteSql throws the below error. I don't know which table throws this
>>> error out of 78 table. i am trying to find the table cause the problem.
>>> Meanwhile if anyone faced this kind of problem please help me.
>>>
>>> 2016-10-04 16:06:23,881 WARN [Timer-Driven Process Thread-3]
>>> o.a.n.c.t.ContinuallyRunProcessorTask Administratively Yielding
>>> ExecuteSQL[id=8fef7912-0157-1000-0000-000075a22c67] due to uncaught
>>> Exception: java.lang.IllegalArgumentException: createSchema: Unknown
>>> SQL type 1111 cannot be converted to Avro type
>>>
>>> 2016-10-04 16:06:23,882 WARN [Timer-Driven Process Thread-3]
>>> o.a.n.c.t.ContinuallyRunProcessorTask
>>>
>>> java.lang.IllegalArgumentException: createSchema: Unknown SQL type 1111
>>> cannot be converted to Avro type
>>>
>>> at org.apache.nifi.processors.standard.util.JdbcCommon.createSchema(JdbcCommon.java:280)
>>> ~[na:na]
>>>
>>> at org.apache.nifi.processors.standard.util.JdbcCommon.convertT
>>> oAvroStream(JdbcCommon.java:88) ~[na:na]
>>>
>>> at org.apache.nifi.processors.standard.util.JdbcCommon.convertT
>>> oAvroStream(JdbcCommon.java:83) ~[na:na]
>>>
>>> at org.apache.nifi.processors.standard.util.JdbcCommon.convertT
>>> oAvroStream(JdbcCommon.java:74) ~[na:na]
>>>
>>> at org.apache.nifi.processors.standard.ExecuteSQL$2.process(ExecuteSQL.java:193)
>>> ~[na:na]
>>>
>>> at org.apache.nifi.controller.repository.StandardProcessSession
>>> .write(StandardProcessSession.java:2123) ~[nifi-framework-core-1.0.0.ja
>>> r:1.0.0]
>>>
>>> at org.apache.nifi.processors.standard.ExecuteSQL.onTrigger(ExecuteSQL.java:187)
>>> ~[na:na]
>>>
>>> at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
>>> ~[nifi-api-1.0.0.jar:1.0.0]
>>>
>>> at org.apache.nifi.controller.StandardProcessorNode.onTrigger(S
>>> tandardProcessorNode.java:1064) ~[nifi-framework-core-1.0.0.jar:1.0.0]
>>>
>>> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>>> .call(ContinuallyRunProcessorTask.java:136)
>>> [nifi-framework-core-1.0.0.jar:1.0.0]
>>>
>>> at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask
>>> .call(ContinuallyRunProcessorTask.java:47)
>>> [nifi-framework-core-1.0.0.jar:1.0.0]
>>>
>>> at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingA
>>> gent$1.run(TimerDrivenSchedulingAgent.java:132)
>>> [nifi-framework-core-1.0.0.jar:1.0.0]
>>>
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> [na:1.8.0_91]
>>>
>>> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>>> [na:1.8.0_91]
>>>
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>> tureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_91]
>>>
>>> at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFu
>>> tureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_91]
>>>
>>> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [na:1.8.0_91]
>>>
>>> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [na:1.8.0_91]
>>>
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_91]
>>>
>>> --
>>> Selvam Raman
>>> "லஞ்சம் தவிர்த்து நெஞ்சம் நிமிர்த்து"
>>>
>>>
>

Mime
View raw message