manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Karl Wright <daddy...@gmail.com>
Subject Re: Handle SolrServerException in SolrConnecotor, HttpPoster
Date Wed, 09 Apr 2014 09:59:37 GMT
Hmm.

I looked at the code in order to make this change, and am now puzzled.
Please see my comments in CONNECTORS-920.

Karl


On Wed, Apr 9, 2014 at 5:35 AM, Karl Wright <daddywri@gmail.com> wrote:

> Hi Hiroshi,
>
> I've created CONNECTORS-920 to describe this issue (
> https://issues.apache.org/jira/browse/CONNECTORS-920).  In the future,
> for cases like this, you can create tickets yourself.
>
>
> One point, though, is that SocketTimeoutException is typically thrown when
> Solr fails to respond during the time a document is being posted to it.
> This is likely due to load on the Solr instance.  Given that, it does not
> make much sense to skip the document, but to retry it later.  This is the
> fix I will make.
>
> Karl
>
>
>
> On Tue, Apr 8, 2014 at 11:35 PM, Hiroshi Tatsumi <
> honekichi19@comet.ocn.ne.jp> wrote:
>
>> Hi,
>>
>> I'm using MCF1.5.1 and Solr4.6.1.
>>
>> When I use SolrConnecotor, sometimes SolrServerException occurs.
>>
>> Normally, SolrServerException is caught by HttpPoster, line 950.
>> But in my case, the inner exception of SolrServerException is
>> SocketTimeoutException, not SocketException.
>> So, activityCode is set to failed, then mcf interrupt the crawl process.
>>
>> In this case, I expect that mcf shouldn't interrupt the crawl process.
>> Mcf should skip the invalid file.
>> Could you modify the mcf code, or provide a option?
>>
>> If you could,  I'm glad to being modified in the future version.
>>
>> Regards,
>> Hiroshi Tatsumi
>>
>>
>

Mime
View raw message