nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (NUTCH-1483) Can't crawl filesystem with protocol-file plugin
Date Tue, 04 Nov 2014 21:58:37 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-1483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14196916#comment-14196916
] 

Hudson commented on NUTCH-1483:
-------------------------------

SUCCESS: Integrated in Nutch-trunk #2848 (See [https://builds.apache.org/job/Nutch-trunk/2848/])
NUTCH-1483 (including NUTCH-1879, NUTCH-1880, NUTCH-1885) fix errors related to protocol-file
(snagel: http://svn.apache.org/viewvc/nutch/trunk/?view=rev&rev=1636736)
* /nutch/branches/2.x/CHANGES.txt
* /nutch/branches/2.x/conf/nutch-default.xml
* /nutch/branches/2.x/conf/regex-normalize.xml.template
* /nutch/branches/2.x/src/java/org/apache/nutch/util/URLUtil.java
* /nutch/branches/2.x/src/plugin/protocol-file/src/java/org/apache/nutch/protocol/file/File.java
* /nutch/branches/2.x/src/plugin/urlnormalizer-regex/sample/regex-normalize-default.test
* /nutch/branches/2.x/src/plugin/urlnormalizer-regex/sample/regex-normalize-default.xml
* /nutch/branches/2.x/src/test/org/apache/nutch/util/TestURLUtil.java
* /nutch/trunk/CHANGES.txt
* /nutch/trunk/conf/nutch-default.xml
* /nutch/trunk/conf/regex-normalize.xml.template
* /nutch/trunk/src/java/org/apache/nutch/util/URLUtil.java
* /nutch/trunk/src/plugin/protocol-file/src/java/org/apache/nutch/protocol/file/File.java
* /nutch/trunk/src/plugin/urlnormalizer-regex/sample/regex-normalize-default.test
* /nutch/trunk/src/plugin/urlnormalizer-regex/sample/regex-normalize-default.xml
* /nutch/trunk/src/test/org/apache/nutch/util/TestURLUtil.java


> Can't crawl filesystem with protocol-file plugin
> ------------------------------------------------
>
>                 Key: NUTCH-1483
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1483
>             Project: Nutch
>          Issue Type: Bug
>          Components: protocol
>    Affects Versions: 1.6, 2.1
>         Environment: OpenSUSE 12.1, OpenJDK 1.6.0, HBase 0.90.4
>            Reporter: Rogério Pereira Araújo
>            Priority: Critical
>             Fix For: 2.3, 1.10
>
>         Attachments: TestProtocolFileUrlUri.java
>
>
> I tried to follow the same steps described in this wiki page:
> http://wiki.apache.org/nutch/IntranetDocumentSearch
> I made all required changes on regex-urlfilter.txt and added the following entry in my
seed file:
> file:///home/rogerio/Documents/
> The permissions are ok, I'm running nutch with the same user as folder owner, so nutch
has all the required permissions, unfortunately I'm getting the following error:
> org.apache.nutch.protocol.file.FileError: File Error: 404
>         at org.apache.nutch.protocol.file.File.getProtocolOutput(File.java:105)
>         at org.apache.nutch.fetcher.FetcherReducer$FetcherThread.run(FetcherReducer.java:514)
> fetch of file://home/rogerio/Documents/ failed with: org.apache.nutch.protocol.file.FileError:
File Error: 404
> Why the logs are showing file://home/rogerio/Documents/ instead of file:///home/rogerio/Documents/
???
> Note: The regex-urlfilter entry only works as expected if I add the entry 
> +^file://home/rogerio/Documents/ instead of +^file:///home/rogerio/Documents/ as wiki
says.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message