nifi-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Witt <joe.w...@gmail.com>
Subject Re: Get IOException occasionally
Date Wed, 15 May 2019 12:59:42 GMT
..another possible factor here is that if two different nars, or processors
which use per instance classloading (as the hadoop ones do), then only one
will have its native libs loaded.  Others will fail to load that same
native lib as this is a JVM wide enforced thing.  So if you have multiple
of these in your flow using the same native lib on some runs one processor
could work and on another a different one will work.  This is frustrating
for users for sure.  The solution is to leverage libraries and update
libraries and/or intercept and do it for them such that we given each
native lib a unique name in the JVM on a per classloader basis.  Just
something we need to knock out.

On Wed, May 15, 2019 at 8:54 AM Shawn Weeks <sweeks@weeksconsulting.us>
wrote:

> There are several complications using Snappy but from what I can tell the
> SelectHiveQL Processor isn't trying to use it and instead the Hive JDBC
> Driver is trying to use it. Are you running a specific vendors distribution
> of NiFi? The reason I'm asking is when I go look at the source for the Hive
> JDBC Driver Class org.apache.hive.jdbc.Utils.verifySuccess where the error
> is being raised none of the line numbers are matching up with Apache. It
> may be that the error is actually coming from Hive not NiFi.
>
> Thanks
> Shawn
>
> ´╗┐On 5/14/19, 6:37 PM, "David Yu" <tsunbonyu@gmail.com> wrote:
>
>     Hi all,
>     I need help finding the root cause on this issue.
>     I occasionally get the following exception from SelectHiveQL and
> PutHiveQL
>     processor.
>     Sometimes the queries executed without any issue but next minute this
>     exception could happen.
>     Once it happened, by restarting only the NiFi does not resolve the
> issue.
>     I have to reboot the Linux VM of the node and remove the flow.xml.gz
> file
>     then restart nifi node.
>
>     I am using NIFI version 1.7.1 in cluster mode with total 8 nodes
>
>     Here is the stacktrace
>     2019-05-14 20:37:30,937 ERROR [Timer-Driven Process Thread-6]
>     o.a.nifi.processors.hive.SelectHiveQL
>     SelectHiveQL[id=365c2597-9483-302c-90d0-c7d2d7ce471d] Unable to execute
>     HiveQL select query select max(version) as max_partition, min(version)
> as
>     min_partition from Table_A for
>
> StandardFlowFileRecord[uuid=9253b1ab-b4a2-4547-b695-4dbf5f964ca2,claim=,offset=0,name=done,size=0]
>     due to org.apache.nifi.processor.exception.ProcessException: Error
> during
>     database query or conversion of records.; routing to failure:
>     org.apache.nifi.processor.exception.ProcessException: Error during
> database
>     query or conversion of records.
>     org.apache.nifi.processor.exception.ProcessException: Error during
> database
>     query or conversion of records.
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$2(SelectHiveQL.java:407)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$627/1348460623.process(Unknown
>     Source)
>     at
>
> org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2625)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:395)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$0(SelectHiveQL.java:285)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$622/285174376.execute(Unknown
>     Source)
>     at
>
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:114)
>     at
>
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:285)
>     at
>
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
>     at
>
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
>     at
>
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>     at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
>     Caused by: java.sql.SQLException: java.io.IOException:
>     java.lang.RuntimeException: native snappy library not available
>     at org.apache.hive.jdbc.Utils.verifySuccess(Utils.java:121)
>     at org.apache.hive.jdbc.Utils.verifySuccessWithInfo(Utils.java:109)
>     at
> org.apache.hive.jdbc.HiveQueryResultSet.next(HiveQueryResultSet.java:330)
>     at
>
> org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
>     at
>
> org.apache.commons.dbcp.DelegatingResultSet.next(DelegatingResultSet.java:207)
>     at
>
> org.apache.nifi.util.hive.HiveJdbcCommon.convertToCsvStream(HiveJdbcCommon.java:355)
>     at
>
> org.apache.nifi.util.hive.HiveJdbcCommon.convertToCsvStream(HiveJdbcCommon.java:322)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.lambda$onTrigger$2(SelectHiveQL.java:401)
>     ... 18 common frames omitted
>     2019-05-14 20:37:30,938 ERROR [Timer-Driven Process Thread-6]
>     o.a.nifi.processors.hive.SelectHiveQL
>     SelectHiveQL[id=365c2597-9483-302c-90d0-c7d2d7ce471d]
>
> org.apache.nifi.processors.hive.SelectHiveQL$$Lambda$622/285174376@54eb300d
>     failed to process due to
>     org.apache.nifi.processor.exception.FlowFileHandlingException:
>
> StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim
>     [resourceClaim=StandardResourceClaim[id=1557687270864-64540,
>     container=default, section=28], offset=81777,
>     length=280],offset=0,name=done,size=280] transfer relationship not
>     specified; rolling back session:
>     org.apache.nifi.processor.exception.FlowFileHandlingException:
>
> StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim
>     [resourceClaim=StandardResourceClaim[id=1557687270864-64540,
>     container=default, section=28], offset=81777,
>     length=280],offset=0,name=done,size=280] transfer relationship not
> specified
>     org.apache.nifi.processor.exception.FlowFileHandlingException:
>
> StandardFlowFileRecord[uuid=e2688819-edc9-406a-b5da-050ac072cd43,claim=StandardContentClaim
>     [resourceClaim=StandardResourceClaim[id=1557687270864-64540,
>     container=default, section=28], offset=81777,
>     length=280],offset=0,name=done,size=280] transfer relationship not
> specified
>     at
>
> org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:264)
>     at
>
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:335)
>     at
>
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:115)
>     at
>
> org.apache.nifi.processor.util.pattern.PartialFunctions.onTrigger(PartialFunctions.java:106)
>     at
>
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:285)
>     at
>
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
>     at
>
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
>     at
>
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
>     at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>     at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>     at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
>     at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>     at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>     at java.lang.Thread.run(Thread.java:745)
>
>
>     Thanks
>     David
>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message