nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <>
Subject Re: Failing Nightly Builds
Date Tue, 08 Jan 2013 03:44:57 GMT
Hi Tejas,

The Jenkins seems to have had a reboot (or something of this nature) around
Christmas. I need to be honest and say that I don't know the source of the
Saying that, Hadoop (and other technologies) can also be a funny bugger
sometimes when it comes to security, proxy, inet addresses. We've witnessed
this a good few times over in Gora where HBase/Hadoop servers fail to
initiate due to intermittent proxy settings/problems... I don't know enough
about the code to provide a definitive answer.

For the time being, I am happy to progress with integrating tests and minor
tickets into Nutch, however I think we really ought to sort the source or
this stack trace regardless of the fact it is on Jenkins.

Maybe we should head over to hadoop general?

I thought I would let this thread stew a while before pouncing on it again.


On Mon, Jan 7, 2013 at 7:22 PM, Tejas Patil <>wrote:

> Hi Lewis,
> These test cases pass on my machine (i guess on yours' too). Had it been
> related to Hadoop API then tests must fail everywhere. What is different
> about the setup where the nightly builds are executed ?
> Thanks,
> Tejas Patil
> On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney <
>> wrote:
>> Hi All,
>> An update to this issue then...
>> The failing tests indicate an addition (security?) feature which makes
>> the tests fail on Jenkins as it results in the following stack traces for
>> the following tests
>> TestCrawlDBMerger
>> TestGenerator
>> TestInjector
>> TestLinkDBMerger
>> TestFetcher
>> TestSegmentMerger
>> Same every time.
>> Currently I am not sure how we can work around this, however I suspect
>> that we need to use some other aspect of the Hadoop API in all calls to
>> obtain socket addresses from servers.
>> Any ideas?
>> -s: -s
>> 	at
>> 	at org.apache.hadoop.mapred.JobClient$
>> 	at org.apache.hadoop.mapred.JobClient$
>> 	at Method)
>> 	at
>> 	at
>> 	at org.apache.hadoop.mapred.JobClient.submitJobInternal(
>> 	at org.apache.hadoop.mapred.JobClient.submitJob(
>> 	at org.apache.hadoop.mapred.JobClient.runJob(
>> 	at org.apache.nutch.crawl.Generator.generate(
>> 	at org.apache.nutch.crawl.Generator.generate(
>> 	at org.apache.nutch.crawl.TestGenerator.generateFetchlist(
>> 	at org.apache.nutch.crawl.TestGenerator.testFilter(
>> --
>> *Lewis*


View raw message