lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hoss Man (Commented) (JIRA)" <>
Subject [jira] [Commented] (SOLR-2804) Logging error causes entire DIH process to fail
Date Wed, 04 Jan 2012 00:42:40 GMT


Hoss Man commented on SOLR-2804:

I've only skimmed the patch, but it seems like overkill for the bug as i understand it --
but maybe i'm just missing something...

*Has anyone actually seen this bug reproduce in any situation where they _weren't_ using DIH
multithreaded mode?*

bq. It blocks all users who index in multiple threads whether via SolrJ or DIH

That should not be the case at all for SOlrJ users and people using DIH in single threaded
mode -- every UpdateProcessorFactory (including the LogUpdateprocessorFactory) creates new
UpdateProcessor instances for every update request -- so no concurrency protection should
be required for multiple threads processing concurrent requests individually.  A simple NamedList
to collect the results to be logged _for each request_ should be completely sufficient.

Based on the comments so far, it sounds like the bug is multiple threads in DIH attempting
to update the same NamedList concurrently -- which is most certainly not safe for DIH to do
(NamedList has never been, nor been documented to suggest it might be, thread safe)

Changing LogUpdateprocessorFactory to add concurrency boundaries to something that is already
suppose to only be single thread access seems like it will only slow down all the use cases
that are already doing the right thing and working fine -- let's fix the concurrency issue
in the place where there is concurrency code: figure out when/why/where DIH is letting multiple
threads manipulate the toLog NamedList and fix that.

> Logging error causes entire DIH process to fail
> -----------------------------------------------
>                 Key: SOLR-2804
>                 URL:
>             Project: Solr
>          Issue Type: Bug
>          Components: contrib - DataImportHandler
>    Affects Versions: 4.0
>         Environment: java version "1.6.0_26"
> Java(TM) SE Runtime Environment (build 1.6.0_26-b03-384-10M3425)
> Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02-384, mixed mode)
> Model Name:	MacBook Pro
>   Model Identifier:	MacBookPro8,2
>   Processor Name:	Intel Core i7
>   Processor Speed:	2.2 GHz
>   Number of Processors:	1
>   Total Number of Cores:	4
>   L2 Cache (per Core):	256 KB
>   L3 Cache:	6 MB
>   Memory:	4 GB
> System Software Overview:
>   System Version:	Mac OS X 10.6.8 (10K549)
>   Kernel Version:	Darwin 10.8.0
>            Reporter: Pulkit Singhal
>              Labels: dih
>         Attachments: SOLR-2804-3x.patch, SOLR-2804.patch
>   Original Estimate: 48h
>  Remaining Estimate: 48h
> SEVERE: Full Import failed:java.lang.ClassCastException:
> java.util.ArrayList cannot be cast to java.lang.String
>        at org.apache.solr.common.util.NamedList.getName(
>        at org.apache.solr.common.util.NamedList.toString(
>        at java.lang.String.valueOf(
>        at java.lang.StringBuilder.append(
>        at org.apache.solr.update.processor.LogUpdateProcessor.finish(
>        at org.apache.solr.handler.dataimport.SolrWriter.close(
>        at org.apache.solr.handler.dataimport.DocBuilder.execute(
>        at org.apache.solr.handler.dataimport.DataImporter.doFullImport(
>        at org.apache.solr.handler.dataimport.DataImporter.runCmd(
>        at org.apache.solr.handler.dataimport.DataImporter$

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message