qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robbie Gemmell (Resolved) (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (QPID-3646) Java Broker becomes unresponsive after BDB Null Pointer Exception encountered
Date Sun, 19 Feb 2012 17:26:36 GMT

     [ https://issues.apache.org/jira/browse/QPID-3646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Robbie Gemmell resolved QPID-3646.
----------------------------------

    Resolution: Not A Problem

Resolving this, as per the Oracle forum there is little we can do about the BDB NPE except
put it down to magic, and I believe there were additional circumstances we later became aware
of around the unresponsiveness of that broker that mean we can be assured it had nothing to
do with the NPE.
                
> Java Broker becomes unresponsive after BDB Null Pointer Exception encountered
> -----------------------------------------------------------------------------
>
>                 Key: QPID-3646
>                 URL: https://issues.apache.org/jira/browse/QPID-3646
>             Project: Qpid
>          Issue Type: Bug
>          Components: Java Broker, Java Broker BDB Store, Java Client
>    Affects Versions: 0.5
>            Reporter: Andrew MacBean
>
> Java Broker becomes unresponsive after BDB Null Pointer Exception encountered in the
logs.  System encountered disk space issues previously but these were not thought to have
affected BDB, only QPID logs. The BDBStore was stored on a different file system that was
healthy.
> The error was observed and then later on clients had issues connecting to the broker
and it was required to be restarted.  The NPE seemed to be thrown by a BDB cleaner thread
that had an invalid offset.
> The BDB Store is version: 4.0.103
> Recorded Timeline of Events
> 1) Nov 23 12:40:57 
> The filesystem where QPID code resides and the logs are written gets filled up , due
to another process, and a message is observed in QPID log:
> log4j:ERROR Failed to flog4j:ERROR Failed to flush writer,
> java.io.IOException: No space left on device
> Disk space issue was then resolved by 1300.
> 2) Nov 23 15:27:41
> The following message appears in the QPID log:
> <DaemonThread name="Cleaner-1"/> caught exception: java.lang.NullPointerException
> java.lang.NullPointerException at com.sleepycat.je.cleaner.OffsetList$Segment.get(OffsetList.java:192)
>             at com.sleepycat.je.cleaner.OffsetList.contains(OffsetList.java:135)
>             at com.sleepycat.je.cleaner.TrackedFileSummary.containsObsoleteOffset(TrackedFileSummary.java:169)
>             at com.sleepycat.je.cleaner.FileProcessor.processFile(FileProcessor.java:479)
>             at com.sleepycat.je.cleaner.FileProcessor.doClean(FileProcessor.java:241)
>             at com.sleepycat.je.cleaner.FileProcessor.onWakeup(FileProcessor.java:140)
>             at com.sleepycat.je.utilint.DaemonThread.run(DaemonThread.java:160)
>             at java.lang.Thread.run(Thread.java:662)
> 3) Around Nov 23 16:30 
> Clients start reporting QPID is not responding and report the following errors:
> org.apache.qpid.client.JMSAMQException: Failed to commit: Server did not respond in a
timely fashion [error code 408: Request Timeout]
> and
> Error creating session: org.apache.qpid.AMQTimeoutException: Server did not respond in
a timely fashion [error code 408: Request Timeout]
> Errors in qpid log seen:
> 2011-11-23 16:29:12,715 ERROR [SocketAcceptorIoProcessor-0.3] protocol.AMQPFastProtocolHandler
(AMQPFastProtocolHandler.java:223) - IOException caught in/169.93.5.123:56138(notifier_ods),
session closed implictly: java.io.IOException: Connection reset by peer
> and 
> 2011-11-23 16:33:41,495 WARN  [Queue-housekeeping-notifier] server.AMQChannel (AMQChannel.java:1071)
- IDLE TRANSACTION ALERT [con:1,069(notifier_ods@/169.93.5.123:55879/notifier)/ch:3]  270932
ms
> QPID Management Console is frozen showing no updates 
> QPID Jconsole doesn't show anything wrong  
> 4) Nov 23 17:15:26
> QPID is bounced and no further issues are observed.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message