nifi-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthew Clarke <matt.clarke....@gmail.com>
Subject Re: Starting Nifi encounters "unexpected error"
Date Wed, 18 May 2016 01:06:56 GMT
Out of memory errors in your app log indicate insufficient heap in your
nifi JVM to run your workflow/dataflow. Not knowing what your workflow
looks like, I can not suggest what may be using up your heap. However, if
your server has the memory capacity, you can increase the size of your
NiFi's heap in the bootstrap.conf file. The defaults if you have not
changed them are set to 512 MB. While defaults are great for simple flows,
it is not adequate for larger dataflows and data volumes.
On May 16, 2016 6:53 PM, "Keith Lim" <Keith.Lim@ds-iq.com> wrote:

> Thanks Andy,
>
>
> There was an out-of-memory error which I didn't mention in the log.  I
> decided to clear out the 4 repository directories, thinking that I may have
> to redo the workflow again.  I was pleasantly surprised to see when it
> started up, all my  workflow intact.
>
> Good to know where the actual workflow is stored.
>
>
> Thanks,
> Keith
>
>
> ------------------------------
> *From:* Andy LoPresto <alopresto@apache.org>
> *Sent:* Monday, May 16, 2016 3:35 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Starting Nifi encounters "unexpected error"
>
> Keith,
>
> Can you check the <install dir>/logs/nifi-bootstrap.log and <install
> dir>/logs/nifi-app.log files to see if there is an error explaining why the
> app does not start up?
>
> The workflow itself is saved as <install dir>/conf/flow.xml.gz. I
> understand this terminology can be confusing — a “flowfile” is the atomic
> unit of data in NiFi, and references the associated content and provenance
> records as it passes through the data flow. The data flow file (what you
> called your workflow) is not a “flowfile” and is stored at the location
> above. It is a (compressed) XML representation of the data flow.
>
> Because you deleted the content_repository, you will not be able to resume
> operation on the data that was queued or being processed at the time the
> server went down. You may still have flowfiles marked for processing in the
> flowfile_repository, and you may have to clear those out. In the future,
> don’t delete the repositories when recovering from a server failure unless
> further debugging demonstrates it is absolutely required. NiFi is fairly
> resilient about resuming processing after the machine is restored.
>
> Let us know if this does not help.
>
>
> Andy LoPresto
> alopresto@apache.org
> *alopresto.apache@gmail.com <alopresto.apache@gmail.com>*
> PGP Fingerprint: 70EC B3E5 98A6 5A3F D3C4  BACE 3C6E F65B 2F7D EF69
>
> On May 16, 2016, at 2:31 PM, Keith Lim <Keith.Lim@ds-iq.com> wrote:
>
> My Windows server was restarted and when I started Nifi again, I can
> longer browse to it :
>
> I have done lots of work on a workflow, is there anyway I can retrieve
> that?
> I have deleted the  content_repository and database_repository, but I have
> a feeling that my work is still in the flowfile_repository.
> Should I delete the provenance_repository?   How can I extract my workflow
> from the flowfile_repository?
>
> Thanks,
> Keith
>
> <pastedImage.png>
>
>
>
>
>
> ------------------------------
> *From:* Keith Lim <Keith.Lim@ds-iq.com>
> *Sent:* Wednesday, May 04, 2016 2:16 PM
> *To:* users@nifi.apache.org
> *Subject:* RE: Starting Nifi encounters "unexpected error"
>
> Thanks Matt, that works.
>
>
> *From:* Matt Gilman [mailto:matt.c.gilman@gmail.com
> <matt.c.gilman@gmail.com>]
> *Sent:* Wednesday, May 4, 2016 1:59 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Starting Nifi encounters "unexpected error"
>
>
> Keith,
>
>
> If you restart NiFi again it doesn't help, correct?
>
>
> Assuming this is the case, can you trying deleting the users database from
> database_repository directory and restart again. The database will be
> created during startup so if it is in a bad state it should be re-created.
> The database is simply a cache of users for the system.
>
>
> Matt
>
>
> On Wed, May 4, 2016 at 4:51 PM, Keith Lim <Keith.Lim@ds-iq.com> wrote:
>
> Thanks Matt, the below is when I browsed to loca nifi url.  In the last
> working session of nifi, I was configuring an ExecuteSql processor before
> it blue-d screen.  Could it have got into a bad state?   How do I get out
> of it?
>
>
> Thanks,
> Keith
>
>
>
>
> *This is from nifi-app.log*
> 2016-05-04 13:43:13,198 INFO [NiFi Web Server-69]
> c.s.j.s.i.application.WebApplicationImpl Initiating Jersey application,
> version 'Jersey: 1.19 02/11/2015 03:25 AM'
> 2016-05-04 13:43:13,839 WARN [NiFi Web Server-22]
> org.eclipse.jetty.servlet.ServletHandler /nifi-api/controller/authorities
> java.lang.NullPointerException: null
>                 at
> org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
> ~[nifi-web-security-0.6.0.jar:0.6.0]
>                 at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
> ~[nifi-web-security-0.6.0.jar:0.6.0]
>                 at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
>                 at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38)
> ~[classes/:na]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52)
> ~[classes/:na]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> [jetty-security-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at org.eclipse.jetty.server.Server.handle(Server.java:499)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> [jetty-io-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
> 2016-05-04 13:43:13,839 WARN [NiFi Web Server-21]
> org.eclipse.jetty.servlet.ServletHandler /nifi-api/controller/identity
> java.lang.NullPointerException: null
>                 at
> org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
> ~[nifi-web-security-0.6.0.jar:0.6.0]
>                 at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
> ~[nifi-web-security-0.6.0.jar:0.6.0]
>                 at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> ~[spring-security-web-4.0.3.RELEASE.jar:4.0.3.RELEASE]
>                 at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
>                 at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> ~[spring-web-4.2.4.RELEASE.jar:4.2.4.RELEASE]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38)
> ~[classes/:na]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52)
> ~[classes/:na]
>                 at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> ~[jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> [jetty-security-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> [jetty-servlet-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at org.eclipse.jetty.server.Server.handle(Server.java:499)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> [jetty-server-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> [jetty-io-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> [jetty-util-9.2.11.v20150529.jar:9.2.11.v20150529]
>                 at java.lang.Thread.run(Thread.java:745) [na:1.8.0_77]
>
>
> *This is from nifi-user.log*
> 2016-05-04 13:43:13,774 INFO [NiFi Web Server-69]
> org.apache.nifi.web.filter.RequestLogger Attempting request for (<no user
> found>) POST http://localhost:8080/nifi-api/access/kerberos (source ip:
> 127.0.0.1)
> 2016-05-04 13:43:13,819 INFO [NiFi Web Server-69] o.a.n.w.a.c.IllegalStateExceptionMapper
> java.lang.IllegalStateException: Access tokens are only issued over
> HTTPS.. Returning Conflict response.
>
>
>
>
> *From:* Matt Gilman [mailto:matt.c.gilman@gmail.com]
> *Sent:* Wednesday, May 4, 2016 1:22 PM
> *To:* users@nifi.apache.org
> *Subject:* Re: Starting Nifi encounters "unexpected error"
>
>
> Keith,
>
>
> Were there any errors in any of the log's (nifi-app.log or nifi-user.log)
> before shutdown or during start up. I'm not 100% sure what version your
> running but it appears it's having trouble loading the Anonymous user. This
> user is created during startup, stored in the users database, and should
> always be available. Any additional errors or details would be helpful.
> Thanks.
>
>
> Matt
>
>
> On Wed, May 4, 2016 at 4:05 PM, Keith Lim <Keith.Lim@ds-iq.com> wrote:
>
> My windows machine blue-d screen and after reboot I restarted nifi only to
> get the following error when I browse to nifi (http://localhost:8080/nifi/
> )
> I can do the long route or cleaning up the nifi folder and rebuilding it,
> but want to know if there is a surgical way to remove this issue.
>
>
> Thanks,
> Keith
>
>
> An unexpected error has occurred
> <html> <head> <meta http-equiv="Content-Type" content="text/html;
> charset=UTF-8"/> <title>Error 500 Server Error</title> </head>
> <body><h2>HTTP ERROR 500</h2> <p>Problem accessing
> /nifi-api/controller/authorities. Reason: <pre> Server
> Error</pre></p><h3>Caused by:</h3><pre>java.lang.NullPointerException
at
> org.apache.nifi.web.security.anonymous.NiFiAnonymousUserFilter.createAuthentication(NiFiAnonymousUserFilter.java:58)
> at
> org.springframework.security.web.authentication.AnonymousAuthenticationFilter.doFilter(AnonymousAuthenticationFilter.java:97)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.apache.nifi.web.security.node.NodeAuthorizedUserFilter.doFilter(NodeAuthorizedUserFilter.java:112)
> at
> org.springframework.security.web.FilterChainProxy$VirtualFilterChain.doFilter(FilterChainProxy.java:330)
> at
> org.springframework.security.web.FilterChainProxy.doFilterInternal(FilterChainProxy.java:213)
> at
> org.springframework.security.web.FilterChainProxy.doFilter(FilterChainProxy.java:176)
> at
> org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
> at
> org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.apache.nifi.web.filter.ThreadLocalFilter.doFilter(ThreadLocalFilter.java:38)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at org.apache.nifi.web.filter.TimerFilter.doFilter(TimerFilter.java:52) at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:585)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:577)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:223)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:110)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
> at org.eclipse.jetty.server.Server.handle(Server.java:499) at
> org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:310) at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
> at
> org.eclipse.jetty.io.AbstractConnection$2.run(AbstractConnection.java:540)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
> at java.lang.Thread.run(Thread.java:745) </pre> <hr><i><small>Powered
by
> Jetty://</small></i><hr/> </body> </html>
>
>
>

Mime
View raw message