nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Lewis John Mcgibbney <lewis.mcgibb...@gmail.com>
Subject Re: Failing Nightly Builds
Date Wed, 09 Jan 2013 03:38:44 GMT
Hi Tejas,

I have no clue on this one I really don't.
I've posted over on user@hadoop... hopefully something will offer itself
forth to us.
In the meantime, thanks for your interest.
Best
Lewis

On Mon, Jan 7, 2013 at 9:32 PM, Tejas Patil <tejas.patil.cs@gmail.com>wrote:

> Hi Lewis,
>
> I feel that this issue might be surrounding "/etc/hosts" file. In [0],
> Dennis Kubes suggested some change to the hosts file. (same thing was
> mentioned in article [1]). In [2], the suggested to check if ssh works
> using hostname and ip.
>
> [0] :
> http://lucene.472066.n3.nabble.com/Nutch-Crawling-error-td612107.html
> [1] : http://www.thegeekstuff.com/2012/02/hadoop-standalone-installation/
> [2] : http://www.mail-archive.com/user@cassandra.apache.org/msg16668.html
>
> Thanks,
> Tejas Patil
>
>
> On Mon, Jan 7, 2013 at 7:44 PM, Lewis John Mcgibbney <
> lewis.mcgibbney@gmail.com> wrote:
>
>> Hi Tejas,
>>
>> The Jenkins seems to have had a reboot (or something of this nature)
>> around Christmas. I need to be honest and say that I don't know the source
>> of the problem.
>> Saying that, Hadoop (and other technologies) can also be a funny bugger
>> sometimes when it comes to security, proxy, inet addresses. We've witnessed
>> this a good few times over in Gora where HBase/Hadoop servers fail to
>> initiate due to intermittent proxy settings/problems... I don't know enough
>> about the code to provide a definitive answer.
>>
>> For the time being, I am happy to progress with integrating tests and
>> minor tickets into Nutch, however I think we really ought to sort the
>> source or this stack trace regardless of the fact it is on Jenkins.
>>
>> Maybe we should head over to hadoop general?
>>
>> I thought I would let this thread stew a while before pouncing on it
>> again.
>>
>> Lewis
>>
>> On Mon, Jan 7, 2013 at 7:22 PM, Tejas Patil <tejas.patil.cs@gmail.com>wrote:
>>
>>> Hi Lewis,
>>>
>>> These test cases pass on my machine (i guess on yours' too). Had it been
>>> related to Hadoop API then tests must fail everywhere. What is different
>>> about the setup where the nightly builds are executed ?
>>>
>>> Thanks,
>>> Tejas Patil
>>>
>>>
>>> On Mon, Jan 7, 2013 at 3:24 PM, Lewis John Mcgibbney <
>>> lewis.mcgibbney@gmail.com> wrote:
>>>
>>>> Hi All,
>>>>
>>>> An update to this issue then...
>>>>
>>>> The failing tests indicate an addition (security?) feature which makes
>>>> the tests fail on Jenkins as it results in the following stack traces for
>>>> the following tests
>>>>
>>>> TestCrawlDBMerger
>>>> TestGenerator
>>>> TestInjector
>>>> TestLinkDBMerger
>>>> TestFetcher
>>>> TestSegmentMerger
>>>>
>>>> Same every time.
>>>> Currently I am not sure how we can work around this, however I suspect
>>>> that we need to use some other aspect of the Hadoop API in all calls to
>>>> obtain socket addresses from servers.
>>>> Any ideas?
>>>>
>>>> java.net.UnknownHostException: -s: -s
>>>> 	at java.net.InetAddress.getLocalHost(InetAddress.java:1354)
>>>> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:936)
>>>> 	at org.apache.hadoop.mapred.JobClient$2.run(JobClient.java:912)
>>>> 	at java.security.AccessController.doPrivileged(Native Method)
>>>> 	at javax.security.auth.Subject.doAs(Subject.java:396)
>>>> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1136)
>>>> 	at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:912)
>>>> 	at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:886)
>>>> 	at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:1323)
>>>> 	at org.apache.nutch.crawl.Generator.generate(Generator.java:551)
>>>> 	at org.apache.nutch.crawl.Generator.generate(Generator.java:465)
>>>> 	at org.apache.nutch.crawl.TestGenerator.generateFetchlist(TestGenerator.java:313)
>>>> 	at org.apache.nutch.crawl.TestGenerator.testFilter(TestGenerator.java:259)
>>>>
>>>>
>>>>
>>>> --
>>>> *Lewis*
>>>>
>>>
>>>
>>
>>
>> --
>> *Lewis*
>>
>
>


-- 
*Lewis*

Mime
View raw message