jackrabbit-oak-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chetan Mehrotra (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (OAK-2560) SegmentStore deadlock on shutdown
Date Sun, 15 Mar 2015 11:31:38 GMT

    [ https://issues.apache.org/jira/browse/OAK-2560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14362350#comment-14362350
] 

Chetan Mehrotra commented on OAK-2560:
--------------------------------------

bq. Oak#createContentRepository is the wrong place to hook up the async index updater and
2) the async indexer needs a way so we can preemptively interrupt it.

+1

One way would be to stop registering the AsyncIndexer in Oak#createContentRepository at least
for OSGi env. In OSGi env have an independent component which registers the indexer and has
a direct dependency on NodeStore service. This way when NodeStore is unregistered it would
deactivate the indexer. 

Further in indexer where we handle the index update callback we can throw exception if we
got the signal for node store getting closed or we hold on the deactivation untill current
run of indexer gets complete.

> SegmentStore deadlock on shutdown
> ---------------------------------
>
>                 Key: OAK-2560
>                 URL: https://issues.apache.org/jira/browse/OAK-2560
>             Project: Jackrabbit Oak
>          Issue Type: Bug
>          Components: segmentmk
>            Reporter: Alex Parvulescu
>            Assignee: Alex Parvulescu
>             Fix For: 1.2
>
>
> We've ran into a deadlock in the Oak SegmentNodeStore between
> _org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.deactivate(SegmentNodeStoreService.java:300)_
and _ org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:120)_
> This was seen on Oak 1.0.11.
> {code}
> Java stack information for the threads listed above 
>  
> "pool-9-thread-5":
> 	at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.readSegment(FileStore.java:656)
> 	- waiting to lock <0x0000000680024468> (a org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentTracker.getSegment(SegmentTracker.java:120)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentId.getSegment(SegmentId.java:101)
> 	- locked <0x00000006b0b7ae28> (a org.apache.jackrabbit.oak.plugins.segment.SegmentId)
> 	at org.apache.jackrabbit.oak.plugins.segment.Record.getSegment(Record.java:82)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.size(MapRecord.java:128)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:462)
> 	- locked <0x0000000680c634a8> (a org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMapBucket(SegmentWriter.java:460)
> 	- locked <0x0000000680c634a8> (a org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeMap(SegmentWriter.java:660)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1020)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:1022)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1003)
> 	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:994)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter$2.childNodeChanged(SegmentWriter.java:1003)
> 	at org.apache.jackrabbit.oak.plugins.memory.ModifiedNodeState.compareAgainstBaseState(ModifiedNodeState.java:396)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.writeNode(SegmentWriter.java:994)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.getNodeState(SegmentNodeBuilder.java:91)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:77)
> 	at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.updated(MemoryNodeBuilder.java:204)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeBuilder.updated(SegmentNodeBuilder.java:73)
> 	at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setChildNode(MemoryNodeBuilder.java:331)
> 	at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.setChildNode(MemoryNodeBuilder.java:323)
> 	at org.apache.jackrabbit.oak.plugins.memory.MemoryNodeBuilder.child(MemoryNodeBuilder.java:311)
> 	at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy.insert(ContentMirrorStoreStrategy.java:112)
> 	at org.apache.jackrabbit.oak.plugins.index.property.strategy.ContentMirrorStoreStrategy.update(ContentMirrorStoreStrategy.java:81)
> 	at org.apache.jackrabbit.oak.plugins.index.property.PropertyIndexEditor.leave(PropertyIndexEditor.java:261)
> 	at org.apache.jackrabbit.oak.spi.commit.CompositeEditor.leave(CompositeEditor.java:74)
> 	at org.apache.jackrabbit.oak.spi.commit.VisibleEditor.leave(VisibleEditor.java:63)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:130)
> 	at org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState.compareAgainstEmptyState(EmptyNodeState.java:160)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:391)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:125)
> 	at org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState.compareAgainstEmptyState(EmptyNodeState.java:160)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:391)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:125)
> 	at org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState.compareAgainstEmptyState(EmptyNodeState.java:160)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:391)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:125)
> 	at org.apache.jackrabbit.oak.plugins.memory.EmptyNodeState.compareAgainstEmptyState(EmptyNodeState.java:160)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:391)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeAdded(EditorDiff.java:125)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeAdded(MapRecord.java:383)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$3.childNodeAdded(MapRecord.java:425)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:483)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:422)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:380)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:540)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:389)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$3.childNodeChanged(MapRecord.java:430)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:473)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:422)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:380)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:540)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:389)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$3.childNodeChanged(MapRecord.java:430)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:473)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:422)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:380)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:540)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.childNodeChanged(EditorDiff.java:148)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord$2.childNodeChanged(MapRecord.java:389)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:473)
> 	at org.apache.jackrabbit.oak.plugins.segment.MapRecord.compare(MapRecord.java:380)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeState.compareAgainstBaseState(SegmentNodeState.java:540)
> 	at org.apache.jackrabbit.oak.spi.commit.EditorDiff.process(EditorDiff.java:52)
> 	at org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.updateIndex(AsyncIndexUpdate.java:354)
> 	at org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate.run(AsyncIndexUpdate.java:300)
> 	- locked <0x000000068afa00e8> (a org.apache.jackrabbit.oak.plugins.index.AsyncIndexUpdate)
> 	at org.apache.sling.commons.scheduler.impl.QuartzJobExecutor.execute(QuartzJobExecutor.java:105)
> 	at org.quartz.core.JobRunShell.run(JobRunShell.java:207)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> 	at java.lang.Thread.run(Thread.java:744)
>  "FelixStartLevel":
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentWriter.flush(SegmentWriter.java:167)
> 	- waiting to lock <0x0000000680c634a8> (a org.apache.jackrabbit.oak.plugins.segment.SegmentWriter)
> 	at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.flush(FileStore.java:431)
> 	- locked <0x000000068461ecb8> (a java.util.concurrent.atomic.AtomicReference)
> 	at org.apache.jackrabbit.oak.plugins.segment.file.FileStore.close(FileStore.java:582)
> 	- locked <0x0000000680024468> (a org.apache.jackrabbit.oak.plugins.segment.file.FileStore)
> 	at org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService.deactivate(SegmentNodeStoreService.java:300)
> 	- locked <0x00000006800243c8> (a org.apache.jackrabbit.oak.plugins.segment.SegmentNodeStoreService)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at org.apache.felix.scr.impl.helper.BaseMethod.invokeMethod(BaseMethod.java:231)
> 	at org.apache.felix.scr.impl.helper.BaseMethod.access$500(BaseMethod.java:39)
> 	at org.apache.felix.scr.impl.helper.BaseMethod$Resolved.invoke(BaseMethod.java:624)
> 	at org.apache.felix.scr.impl.helper.BaseMethod.invoke(BaseMethod.java:508)
> 	at org.apache.felix.scr.impl.helper.ActivateMethod.invoke(ActivateMethod.java:149)
> 	at org.apache.felix.scr.impl.manager.SingleComponentManager.disposeImplementationObject(SingleComponentManager.java:355)
> 	at org.apache.felix.scr.impl.manager.SingleComponentManager.deleteComponent(SingleComponentManager.java:170)
> 	at org.apache.felix.scr.impl.manager.AbstractComponentManager.doDeactivate(AbstractComponentManager.java:908)
> 	at org.apache.felix.scr.impl.manager.AbstractComponentManager.deactivateInternal(AbstractComponentManager.java:883)
> 	at org.apache.felix.scr.impl.manager.AbstractComponentManager.dispose(AbstractComponentManager.java:580)
> 	at org.apache.felix.scr.impl.config.ConfigurableComponentHolder.disposeComponents(ConfigurableComponentHolder.java:406)
> 	at org.apache.felix.scr.impl.BundleComponentActivator.dispose(BundleComponentActivator.java:335)
> 	at org.apache.felix.scr.impl.Activator.disposeComponents(Activator.java:313)
> 	at org.apache.felix.scr.impl.Activator.access$300(Activator.java:45)
> 	at org.apache.felix.scr.impl.Activator$ScrExtension.destroy(Activator.java:198)
> 	at org.apache.felix.utils.extender.AbstractExtender$2.run(AbstractExtender.java:290)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> 	at org.apache.felix.utils.extender.AbstractExtender.destroyExtension(AbstractExtender.java:312)
> 	at org.apache.felix.utils.extender.AbstractExtender.bundleChanged(AbstractExtender.java:186)
> 	at org.apache.felix.framework.util.EventDispatcher.invokeBundleListenerCallback(EventDispatcher.java:869)
> 	at org.apache.felix.framework.util.EventDispatcher.fireEventImmediately(EventDispatcher.java:790)
> 	at org.apache.felix.framework.util.EventDispatcher.fireBundleEvent(EventDispatcher.java:515)
> 	at org.apache.felix.framework.Felix.fireBundleEvent(Felix.java:4409)
> 	at org.apache.felix.framework.Felix.stopBundle(Felix.java:2526)
> 	at org.apache.felix.framework.Felix.setActiveStartLevel(Felix.java:1315)
> 	at org.apache.felix.framework.FrameworkStartLevelImpl.run(FrameworkStartLevelImpl.java:304)
> 	at java.lang.Thread.run(Thread.java:744)
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message