lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Saurabh Sethi <Saurabh_Se...@symantec.com>
Subject OOM during commit - Lucene 3.6
Date Mon, 30 Sep 2013 18:39:54 GMT
Hi,

I have a method that deletes entries from Lucene and calls IndexWriter.commit() in the end.
This method is called multiple times and each time the batch size varies from 5-20 items.

After some deletions, commit() causes JVM to crash. I am using Lucene 3.6 and JDK is jrockit-jdk-1.6.0_22-R28.1.1-4.0.1-x64.

Following is the partial JVM dump:

2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030) Roots Dump Helper:
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              thread=0x00000000681A2530
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)                 method=org/apache/lucene/index/IndexWriter.prepareCommit(Ljava/util/Map;)V
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              sp=0x000000006AC4D5D0
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              ip=0x000000002088CC8A
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              ref=0x000000006AC4D5A0
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              *ref=0x00000003D914CD81
(interior pointer to 0x00000003D914CD80)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              stackIter=0x000000005F7DFD00
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              refIter=0x000000005F7DFD58
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)              storage=rbp
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030) Roots Processed Thread: "Email Batch
Committer 2" id=251 idx=0x21c tid=21388 lastJavaFrame=0x000000006AC4D2C0
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at jrockit/vm/Allocator.getNewTla(JJ)V(Native
Method)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at jrockit/vm/Allocator.allocObjectOrArray(Allocator.java:354)[optimized]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at java/util/HashMap.<init>(HashMap.java:253)[inlined]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at java/util/HashSet.<init>(HashSet.java:86)[optimized]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/SegmentInfos.files(SegmentInfos.java:836)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexFileDeleter.incRef(IndexFileDeleter.java:480)[inlined]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexFileDeleter.checkpoint(IndexFileDeleter.java:453)[optimized]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter$ReaderPool.commit(IndexWriter.java:648)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.prepareCommit(IndexWriter.java:3383)[optimized]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commitInternal(IndexWriter.java:3491)[optimized]
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commit(IndexWriter.java:3473)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at org/apache/lucene/index/IndexWriter.commit(IndexWriter.java:3457)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/TextIndexWriter.commit(TextIndexWriter.java:348)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.flush(LocalTextIndexImpl.java:483)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.commitDeletes(LocalTextIndexImpl.java:369)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/LocalTextIndexImpl.deleteByIDs(LocalTextIndexImpl.java:280)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/common/textengine/TextHelper.removeEsaDocIDs(TextHelper.java:298)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/indexer/EmailBatchCommitter.deleteDerivedFTDocs(EmailBatchCommitter.java:2004)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)
2013-09-30 02:39:52,467 INFO  [STDOUT] (indexer:46030)     at com/teneo/esa/indexer/EmailBatchCommitter.commitPartitionDocuments(EmailBatchCommitter.java:1291)[optimized]

Would appreciate any help on this.

Thanks,
Saurabh

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message