nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Markus Jelsma (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (NUTCH-1647) protocol-http throws unzipBestEffort returned null for some pages
Date Fri, 13 Jun 2014 08:00:20 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-1647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14030357#comment-14030357
] 

Markus Jelsma edited comment on NUTCH-1647 at 6/13/14 7:58 AM:
---------------------------------------------------------------

I am sorry to say this issue has not been resolved.

{code}
$ bin/nutch indexchecker http://www.provinciegroningen.nl/informatiebalie/publicaties
fetching: http://www.provinciegroningen.nl/informatiebalie/publicaties
Fetch failed with protocol status: exception(16), lastModified=0: java.io.IOException: unzipBestEffort
returned null
{code}

{code}
2014-06-13 09:52:35,580 INFO  http.Http - http.proxy.host = null
2014-06-13 09:52:35,580 INFO  http.Http - http.proxy.port = 8080
2014-06-13 09:52:35,580 INFO  http.Http - http.timeout = 12000
2014-06-13 09:52:35,580 INFO  http.Http - http.content.limit = 5242880
2014-06-13 09:52:35,580 INFO  http.Http - http.agent = Mozilla/5.0 (compatible; OpenindexSpider;
+http://www.openindex.io/en/webmasters/spider.html)
2014-06-13 09:52:35,580 INFO  http.Http - http.accept.language = en-us,en-gb,en;q=0.7,*;q=0.3
2014-06-13 09:52:35,580 INFO  http.Http - http.accept = text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2014-06-13 09:52:35,869 ERROR http.Http - Failed to get protocol output
java.io.IOException: unzipBestEffort returned null
        at org.apache.nutch.protocol.http.api.HttpBase.processGzipEncoded(HttpBase.java:370)
        at org.apache.nutch.protocol.http.HttpResponse.<init>(HttpResponse.java:226)
        at org.apache.nutch.protocol.http.Http.getResponse(Http.java:72)
        at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:183)
        at org.apache.nutch.indexer.IndexingFiltersChecker.run(IndexingFiltersChecker.java:86)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.nutch.indexer.IndexingFiltersChecker.main(IndexingFiltersChecker.java:167)
{code}


was (Author: markus17):
I am sorry to say this issue has not been resolved.

{code}
$ bin/nutch indexchecker http://www.provinciegroningen.nl/informatiebalie/publicaties
fetching: http://www.provinciegroningen.nl/informatiebalie/publicaties
Fetch failed with protocol status: exception(16), lastModified=0: java.io.IOException: unzipBestEffort
returned null
{code}

{code}
2014-06-13 09:52:35,580 INFO  http.Http - http.proxy.host = null
2014-06-13 09:52:35,580 INFO  http.Http - http.proxy.port = 8080
2014-06-13 09:52:35,580 INFO  http.Http - http.timeout = 12000
2014-06-13 09:52:35,580 INFO  http.Http - http.content.limit = 5242880
2014-06-13 09:52:35,580 INFO  http.Http - http.agent = Mozilla/5.0 (compatible; OpenindexSpider;
+http://www.openindex.io/en/webmasters/spider.html)
2014-06-13 09:52:35,580 INFO  http.Http - http.accept.language = en-us,en-gb,en;q=0.7,*;q=0.3
2014-06-13 09:52:35,580 INFO  http.Http - http.accept = text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2014-06-13 09:52:35,869 ERROR http.Http - Failed to get protocol output
java.io.IOException: unzipBestEffort returned null
        at org.apache.nutch.protocol.http.api.HttpBase.processGzipEncoded(HttpBase.java:370)
        at org.apache.nutch.protocol.http.HttpResponse.<init>(HttpResponse.java:221)
        at org.apache.nutch.protocol.http.Http.getResponse(Http.java:72)
        at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:183)
        at org.apache.nutch.indexer.IndexingFiltersChecker.run(IndexingFiltersChecker.java:86)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
        at org.apache.nutch.indexer.IndexingFiltersChecker.main(IndexingFiltersChecker.java:167)
{code}

> protocol-http throws unzipBestEffort returned null for some pages
> -----------------------------------------------------------------
>
>                 Key: NUTCH-1647
>                 URL: https://issues.apache.org/jira/browse/NUTCH-1647
>             Project: Nutch
>          Issue Type: Bug
>          Components: protocol
>    Affects Versions: 1.7
>            Reporter: Markus Jelsma
>             Fix For: 1.9
>
>
> bin/nutch indexchecker http://www.provinciegroningen.nl/actueel/dossiers/rwe-centrale
 
> Fetch failed with protocol status: exception(16), lastModified=0: java.io.IOException:
unzipBestEffort returned null
> {code}
> 2013-10-01 13:44:55,612 INFO  http.Http - http.proxy.host = null
> 2013-10-01 13:44:55,612 INFO  http.Http - http.proxy.port = 8080
> 2013-10-01 13:44:55,612 INFO  http.Http - http.timeout = 12000
> 2013-10-01 13:44:55,612 INFO  http.Http - http.content.limit = 5242880
> 2013-10-01 13:44:55,612 INFO  http.Http - http.agent = Mozilla/5.0 (compatible; OpenindexSpider;
+http://www.openindex.io/en/webmasters/spider.html)
> 2013-10-01 13:44:55,612 INFO  http.Http - http.accept.language = en-us,en-gb,en;q=0.7,*;q=0.3
> 2013-10-01 13:44:55,613 INFO  http.Http - http.accept = text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> 2013-10-01 13:44:55,925 ERROR http.Http - Failed to get protocol output
> java.io.IOException: unzipBestEffort returned null
>         at org.apache.nutch.protocol.http.api.HttpBase.processGzipEncoded(HttpBase.java:317)
>         at org.apache.nutch.protocol.http.HttpResponse.<init>(HttpResponse.java:164)
>         at org.apache.nutch.protocol.http.Http.getResponse(Http.java:64)
>         at org.apache.nutch.protocol.http.api.HttpBase.getProtocolOutput(HttpBase.java:140)
>         at org.apache.nutch.indexer.IndexingFiltersChecker.run(IndexingFiltersChecker.java:86)
>         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>         at org.apache.nutch.indexer.IndexingFiltersChecker.main(IndexingFiltersChecker.java:150)
> {code}
> Haven't got a clue yet as to what the exact issue is.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message