nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ferdy Galema <ferdy.gal...@kalooga.com>
Subject Re: Build failed in Jenkins: Nutch-trunk #1651
Date Thu, 03 Nov 2011 12:31:45 GMT
I can't tell what the exact cause is. Because tests run locally fine and 
because the commits since last build succes seem completely unrelated, I 
would say yes this is definitely caused by the Solaris build 
invironment. Unfortunately I'm still a novice in regard to the build 
process so I'm not sure how to tackle this problem. My suggestion is to 
first try a clean rebuild on the build server. (Is it possibly to fire 
off ad hoc build requests?)

On 11/03/2011 12:39 PM, Lewis John Mcgibbney wrote:
> Hi,
>
> If you look at the recently failing Nutch trunk builds, namely #1645, 
> #1650 & #1651, a common denominator is the 
> org.apache.nutch.segment.TestSegmentMerger.testLargeMerge 
> <https://builds.apache.org/job/Nutch-trunk/1651/testReport/org.apache.nutch.segment/TestSegmentMerger/testLargeMerge/>
 
> which failes with the following:
>
> 2011-11-03 04:17:37,478 WARN  mapred.LocalJobRunner (LocalJobRunner.java:run(256)) -
job_local_0001
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_local_0001/attempt_local_0001_m_000000_0/output/file.out
in any of the configured local directories
>
> 	at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
> 	at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
> 	at org.apache.hadoop.mapred.MapOutputFile.getOutputFile(MapOutputFile.java:50)
>
> 	at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:193)
>
> Any clues as to this pattern of failures? Is it possible to nail it 
> down to the Solaris slave build environment?
>
> On Thu, Nov 3, 2011 at 4:18 AM, Apache Jenkins Server 
> <jenkins@builds.apache.org <mailto:jenkins@builds.apache.org>> wrote:
>
>     See <https://builds.apache.org/job/Nutch-trunk/1651/changes>
>
>     Changes:
>
>     [lewismc] commit to make trivial change to working directory
>     configuration for ant sonar target...
>
>     [lewismc] commit to add MySQL driver classpath to sonar analysis
>     target
>
>     [ferdy] NUTCH-1192 Add '/runtime' to svn ignore
>
>     ------------------------------------------
>     [...truncated 2838 lines...]
>     compile:
>         [echo] Compiling plugin: urlfilter-validator
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 1 source file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/classes
>
>     jar:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlfilter-validator/urlfilter-validator.jar
>
>     deps-test:
>
>     deploy:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
>
>     copy-generated-lib:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlfilter-validator
>
>     init:
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/test
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
>     init-plugin:
>
>     deps-jar:
>
>     clean-lib:
>
>     resolve-default:
>     [ivy:resolve] :: loading settings :: file =
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
>     compile:
>         [echo] Compiling plugin: urlmeta
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 2 source files to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/classes
>
>     jar:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlmeta/urlmeta.jar
>
>     deps-test:
>
>     deploy:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
>     copy-generated-lib:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlmeta
>
>     init:
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/test
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
>     init-plugin:
>
>     deps-jar:
>
>     clean-lib:
>
>     resolve-default:
>     [ivy:resolve] :: loading settings :: file =
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
>     compile:
>         [echo] Compiling plugin: urlnormalizer-basic
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 1 source file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/classes
>
>     jar:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-basic/urlnormalizer-basic.jar
>
>     deps-test:
>
>     deploy:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
>     copy-generated-lib:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-basic
>
>     init:
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/test
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
>
>     init-plugin:
>
>     deps-jar:
>
>     clean-lib:
>
>     resolve-default:
>     [ivy:resolve] :: loading settings :: file =
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
>     compile:
>         [echo] Compiling plugin: urlnormalizer-pass
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 1 source file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/classes
>
>     jar:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-pass/urlnormalizer-pass.jar
>
>     deps-test:
>
>     deploy:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
>
>     copy-generated-lib:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-pass
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
>         [copy] Copying 4 files to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/test/data
>
>     init:
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
>     init-plugin:
>
>     deps-jar:
>
>     clean-lib:
>
>     resolve-default:
>     [ivy:resolve] :: loading settings :: file =
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
>     compile:
>         [echo] Compiling plugin: urlnormalizer-regex
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/build-plugin.xml:117:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 1 source file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/classes
>        [javac] Note:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/src/plugin/urlnormalizer-regex/src/java/org/apache/nutch/net/urlnormalizer/regex/RegexURLNormalizer.java
>     uses unchecked or unsafe operations.
>        [javac] Note: Recompile with -Xlint:unchecked for details.
>
>     jar:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/urlnormalizer-regex/urlnormalizer-regex.jar
>
>     deps-test:
>
>     init:
>
>     init-plugin:
>
>     clean-lib:
>
>     resolve-default:
>     [ivy:resolve] :: loading settings :: file =
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/ivy/ivysettings.xml
>
>     compile:
>
>     jar:
>
>     deps-test:
>
>     deploy:
>
>     copy-generated-lib:
>
>     deploy:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
>     copy-generated-lib:
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/plugins/urlnormalizer-regex
>
>     compile:
>
>     job:
>          [jar] Building jar:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/nutch-2011-11-03_04-11-51.job
>
>     resolve-test:
>      [taskdef] Could not load definitions from resource
>     org/sonar/ant/antlib.xml. It could not be found.
>
>     copy-libs:
>
>     compile-core-test:
>        [javac]
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:320:
>     warning: 'includeantruntime' was not set, defaulting to
>     build.sysclasspath=last; set to false for repeatable builds
>        [javac] Compiling 32 source files to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
>        [javac] Note: Some input files use or override a deprecated API.
>        [javac] Note: Recompile with -Xlint:deprecation for details.
>        [javac] Note: Some input files use unchecked or unsafe operations.
>        [javac] Note: Recompile with -Xlint:unchecked for details.
>
>     test-core:
>        [mkdir] Created dir:
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
>         [copy] Copying 90 files to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/data
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
>         [copy] Copying 1 file to
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/test/classes
>        [junit] WARNING: multiple versions of ant detected in path for
>     junit
>        [junit]        
>      jar:file:/export/home/hudson/tools/ant/latest/lib/ant.jar!/org/apache/tools/ant/Project.class
>        [junit]      and
>     jar:file:/zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build/lib/ant-1.6.5.jar!/org/apache/tools/ant/Project.class
>        [junit] Running org.apache.nutch.crawl.TestCrawlDbMerger
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     7.829 sec
>        [junit] Running org.apache.nutch.crawl.TestGenerator
>        [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
>     60.748 sec
>        [junit] Running org.apache.nutch.crawl.TestInjector
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     10.569 sec
>        [junit] Running org.apache.nutch.crawl.TestLinkDbMerger
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     4.033 sec
>        [junit] Running org.apache.nutch.crawl.TestSignatureFactory
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.491 sec
>        [junit] Running org.apache.nutch.fetcher.TestFetcher
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     21.039 sec
>        [junit] Running org.apache.nutch.indexer.TestIndexingFilters
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     1.025 sec
>        [junit] Running org.apache.nutch.metadata.TestMetadata
>        [junit] Tests run: 10, Failures: 0, Errors: 0, Time elapsed:
>     0.198 sec
>        [junit] Running org.apache.nutch.metadata.TestSpellCheckedMetadata
>        [junit] Tests run: 11, Failures: 0, Errors: 0, Time elapsed:
>     3.679 sec
>        [junit] Running org.apache.nutch.net.TestURLFilters
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     1.617 sec
>        [junit] Running org.apache.nutch.net.TestURLNormalizers
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.655 sec
>        [junit] Running org.apache.nutch.parse.TestOutlinkExtractor
>        [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
>     0.06 sec
>        [junit] Running org.apache.nutch.parse.TestParseData
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     0.052 sec
>        [junit] Running org.apache.nutch.parse.TestParseText
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.15 sec
>        [junit] Running org.apache.nutch.parse.TestParserFactory
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     1.077 sec
>        [junit] Running org.apache.nutch.plugin.TestPluginSystem
>        [junit] Tests run: 7, Failures: 0, Errors: 0, Time elapsed:
>     1.485 sec
>        [junit] Running org.apache.nutch.protocol.TestContent
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     0.548 sec
>        [junit] Running org.apache.nutch.protocol.TestProtocolFactory
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     1.634 sec
>        [junit] Running org.apache.nutch.segment.TestSegmentMerger
>        [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed:
>     135.865 sec
>        [junit] Test org.apache.nutch.segment.TestSegmentMerger FAILED
>        [junit] Running org.apache.nutch.util.TestEncodingDetector
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.572 sec
>        [junit] Running org.apache.nutch.util.TestGZIPUtils
>        [junit] Tests run: 4, Failures: 0, Errors: 0, Time elapsed:
>     1.343 sec
>        [junit] Running org.apache.nutch.util.TestNodeWalker
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.283 sec
>        [junit] Running org.apache.nutch.util.TestPrefixStringMatcher
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.109 sec
>        [junit] Running org.apache.nutch.util.TestStringUtil
>        [junit] Tests run: 2, Failures: 0, Errors: 0, Time elapsed:
>     0.055 sec
>        [junit] Running org.apache.nutch.util.TestSuffixStringMatcher
>        [junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed:
>     0.125 sec
>        [junit] Running org.apache.nutch.util.TestURLUtil
>        [junit] Tests run: 5, Failures: 0, Errors: 0, Time elapsed:
>     0.374 sec
>
>     BUILD FAILED
>     /zonestorage/hudson_solaris/home/hudson/hudson-slave/workspace/Nutch-trunk/trunk/build.xml:395:
>     Tests failed!
>
>     Total time: 5 minutes 20 seconds
>     Build step 'Execute shell' marked build as failure
>     Recording test results
>     Publishing Javadoc
>     Updating NUTCH-1192
>
>
>
>
> -- 
> /Lewis/
>

Mime
View raw message