qpid-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Cliff Jansen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (QPID-3256) Application which uses Qpid (in my case Excel) hangs on shutdown
Date Tue, 21 Jun 2011 07:57:47 GMT

    [ https://issues.apache.org/jira/browse/QPID-3256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13052415#comment-13052415
] 

Cliff Jansen commented on QPID-3256:
------------------------------------

The patch was published in the review board (https://reviews.apache.org/r/904/) where there
was a comment regarding dynamic creation and deletion of the map of Qpid Thread objects:


Alan Conway:

Why the pointer? Just declare 

std::map<unsigned, ThreadPrivate::shared_ptr> pQpidThreads;

and let std::map take care of the memory management. 

    Andrew Stitcher:

        I'd also add that we don't use the hungarian variable notation either so the 'p' should
be stripped (possibly from other places too)

    Cliff Jansen:

        The intended logic was that Thread::current() should have defined behavior, even if
called from a static destructor.  Otherwise, you need a way to guarantee the std:map destructor
is never called too soon.

        The original JIRA's deadlock was in the context of a static destructor calling Thread::join().

        On re-examiniation, the use of a non-POD lock mechanism is inconsistent with the stated
goal.  Thank-you for the heads up.  I will work with Steve to address this and the naming
problem.



On yet further examination, even the PODMutex is not guaranteed to have defined behaviour
during the static destructor resolution at process exit or library unload.  The "static deinitialization
order fiasco" has no current solution in the Boost mutex implementation (see http://www.justsoftwaresolutions.co.uk/articles/implementing_mutexes.html)
despite attempts to address.

I see three possible ways forward:

1. Ship with the proposed fix (updated to address the review comments).  It provides improved
functionality, but a known potential random outage.

2. Use a Windows CRITICAL_SECTION as the mutex, and initialize and release it outside the
C++ runtime in DllMain.  This is a common pattern in Windows and is in fact used in the DTC
plugin module in the wcf subtree.

3. Use a spin lock (via interlocked operations) to manage the exit/unload condition.  Such
a spin lock holds no system resources to leak and requires no explicit shutdown.  This is
the mechanism adopted by the Boost shared_ptr implementation of atomic operations.


If anyone has a further suggestion, or a preference for option 1 please let me know.  Otherwise
I will put up examples of options 2 and 3 to the review board.


> Application which uses Qpid (in my case Excel) hangs on shutdown
> ----------------------------------------------------------------
>
>                 Key: QPID-3256
>                 URL: https://issues.apache.org/jira/browse/QPID-3256
>             Project: Qpid
>          Issue Type: Bug
>          Components: C++ Client
>    Affects Versions: 0.8, 0.10
>         Environment: OS: Windows.
> Qpid is assembled as DLL. 
>            Reporter: Eugene
>            Assignee: Steve Huston
>             Fix For: 0.11
>
>         Attachments: qpid-3256.patch
>
>
> Hi All
> I encountered with strange behavior on shutdown when using qpid 0-8 and 0-10. 
> When I use qpid in standalone console-application everything is ok. But when I use qpid
in DLL which is loaded into Excel (as RTD module), Excel hangs on shutdown. 
> I found out that in standalone application on shutdown I have next stack:
> 	qpidclientd.dll!qpid::client::`anonymous namespace'::IOThread::~IOThread()  Line 138
C++
>  	qpidclientd.dll!`qpid::client::`anonymous namespace'::theIO'::`2'::`dynamic atexit
destructor for 'io''()  + 0xd bytes	C++
>  	qpidclientd.dll!_CRT_INIT(void * hDllHandle=0x60080000, unsigned long dwReason=0, void
* lpreserved=0x00000001)  Line 449	C
>  	qpidclientd.dll!__DllMainCRTStartup(void * hDllHandle=0x60080000, unsigned long dwReason=0,
void * lpreserved=0x00000001)  Line 560 + 0x11 bytes	C
>  	qpidclientd.dll!_DllMainCRTStartup(void * hDllHandle=0x60080000, unsigned long dwReason=0,
void * lpreserved=0x00000001)  Line 510 + 0x11 bytes	C
>  	ntdll.dll!77b79960() 	
>  	[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
>  	ntdll.dll!77b9a516() 	
>  	ntdll.dll!77b9a3b8() 	
>  	kernel32.dll!77657363() 	
>  	msvcr90d.dll!__crtExitProcess(int status=0)  Line 732	C
>  	msvcr90d.dll!doexit(int code=0, int quick=0, int retcaller=0)  Line 644 + 0x9 bytes
C
>  	msvcr90d.dll!exit(int code=0)  Line 412 + 0xd bytes	C
>  	Test.exe!__tmainCRTStartup()  Line 599	C
>  	Test.exe!mainCRTStartup()  Line 403	C
>  	kernel32.dll!77653677() 	
>  	ntdll.dll!77b79f02() 	
>  	ntdll.dll!77b79ed5() 	
> And in this state all threads of application have been already terminated. The only thread
is:
> 1	>	21720	Main Thread	Main Thread	qpid::client::`anonymous namespace'::IOThread::~IOThread
Normal	0
> so code from file ConnectionImpl.cpp works well:
> ~IOThread() {
>         std::vector<Thread> threads;
>         {
>             ScopedLock<Mutex> l(threadLock);
>             if (poller_)
>                 poller_->shutdown();
>             t.swap(threads);
>         }
>         for (std::vector<Thread>::iterator i = threads.begin(); i != threads.end();
++i) {
>             i->join();
>         }
>     }
> BUT in Excel I get stack:
> qpidclientd.dll!qpid::client::`anonymous namespace'::IOThread::~IOThread()  Line 130
C++
>  	qpidclientd.dll!`qpid::client::`anonymous namespace'::theIO'::`2'::`dynamic atexit
destructor for 'io''()  + 0xd bytes	C++
>  	qpidclientd.dll!_CRT_INIT(void * hDllHandle=0x07700000, unsigned long dwReason=0, void
* lpreserved=0x00000000)  Line 449	C
>  	qpidclientd.dll!__DllMainCRTStartup(void * hDllHandle=0x07700000, unsigned long dwReason=0,
void * lpreserved=0x00000000)  Line 560 + 0x11 bytes	C
>  	qpidclientd.dll!_DllMainCRTStartup(void * hDllHandle=0x07700000, unsigned long dwReason=0,
void * lpreserved=0x00000000)  Line 510 + 0x11 bytes	C
>  	ntdll.dll!77b79960() 	
>  	[Frames below may be incorrect and/or missing, no symbols loaded for ntdll.dll]	
>  	ntdll.dll!77ba1525() 	
>  	ntdll.dll!77b81231() 	
>  	KernelBase.dll!77281da7() 	
>  	ole32.dll!75bb9562() 	
>  	ole32.dll!75bb9593() 	
>  	ole32.dll!75bb95a7() 	
>  	ole32.dll!75bb98bf() 	
>  	ole32.dll!75bb9805() 	
>  	ole32.dll!75bb9a8c() 	
>  	EXCEL.EXE!2f3811e9() 	
>  	EXCEL.EXE!2f6933e8() 	
>  	EXCEL.EXE!2f32a5af() 	
>  	EXCEL.EXE!2f34894a() 	
>  	EXCEL.EXE!2f670001() 	
>  	MSO.DLL!65bc6ed5() 	
>  	MSO.DLL!65c26a34() 	
>  	MSO.DLL!65c30305() 	
>  	MSO.DLL!65bc910c() 	
>  	MSO.DLL!65c4f420() 	
>  	MSO.DLL!65bbf161() 	
>  	comctl32.dll!7233463d() 	
>  	user32.dll!762971be() 	
>  	user32.dll!76297d31() 	
>  	user32.dll!76297dfa() 	
>  	EXCEL.EXE!2f324572() 	
>  	EXCEL.EXE!2f324534() 	
>  	EXCEL.EXE!2f324441() 	
>  	MSO.DLL!65b78116() 	
>  	MSO.DLL!65ba1fd0() 	
>  	EXCEL.EXE!2f30424b() 	
>  	msvcr90.dll!749936c5() 	
>  	msvcr90.dll!749938b3() 	
>  	msvcr90.dll!749938c5() 	
>  	msvcr90.dll!749ac40c() 	
>  	msvcr90.dll!749b028d() 	
>  	msvcr90.dll!749b04f3() 	
>  	EXCEL.EXE!2f303f0a() 	
>  	kernel32.dll!77653677() 	
>  	ntdll.dll!77b79f02() 	
>  	ntdll.dll!77b79ed5() 	
> And threads:
> 0	 	24016	Worker Thread	_threadstartex	_threadstartex	Normal	0
> 0	>	22928	Main Thread	Main Thread	qpid::client::`anonymous namespace'::IOThread::~IOThread
Normal	0
> 0	 	20224	RPC Thread	RPC Callback Thread	77b5fd21	Normal	0
> 0	 	16492	Worker Thread	Win32 Thread	77b61ed6	Normal	0
> 0	 	19948	Worker Thread	Win32 Thread	77b600ed	Normal	0
> 0	 	20524	Worker Thread	Win32 Thread	77b61ed6	Normal	0
> 0	 	20532	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	21500	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	21848	Worker Thread	Win32 Thread	77b61ed6	Normal	0
> 0	 	22152	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	22164	Worker Thread	Win32 Thread	77b5f8e9	Normal	0
> 0	 	22300	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	22360	Worker Thread	Win32 Thread	77b61ed6	Normal	0
> 0	 	23316	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	23556	Worker Thread	Win32 Thread	77b5f8e9	Normal	0
> 0	 	23700	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	23912	Worker Thread	Win32 Thread	77b5f8e9	Normal	0
> 0	 	24276	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	24308	Worker Thread	Win32 Thread	77b5f861	Normal	0
> 0	 	24424	Worker Thread	Win32 Thread	77b600ed	Normal	0
> 0	 	24452	Worker Thread	Win32 Thread	77b5f8e9	Normal	0
> 0	 	24520	Worker Thread	Win32 Thread	77b61ed6	Normal	0
> As result this code (below) hangs application(Excel):
> ~IOThread() {
>         std::vector<Thread> threads;
>         {
>             ScopedLock<Mutex> l(threadLock);
>             if (poller_)
>                 poller_->shutdown();
>             t.swap(threads);
>         }
>         for (std::vector<Thread>::iterator i = threads.begin(); i != threads.end();
++i) 
>         {
>             i->join();-- APPLICATION HANGS HERE !!!!
>         }
>     }
> I suppose it occurs because of qpid tries to wait of thread completing from _DllMainCRTStartup
which is called with PROCESS_DETACH.
> To work around it I modified function IOThread::sub():
>    void sub() {
> 		std::vector<Thread> threads;
> 		{
>                   ScopedLock<Mutex> l(threadLock);
> 		  --connections; 
> 		
> 		  if (connections == 0){
> 			if (poller_){
> 				poller_->shutdown();
> 				poller_.reset();
> 				t.swap(threads);
> 			}
> 		  }
>                }
> 	       for (std::vector<Thread>::iterator i = threads.begin(); i != threads.end();
++i) {
> 		 i->join();
> 	     }
>          }
> But I don't think it is a good solution.
> Could you help me to solve this problem?
> Thanks

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:dev-subscribe@qpid.apache.org


Mime
View raw message