nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jeff Shafer (JIRA)" <j...@apache.org>
Subject [jira] Issue Comment Edited: (NUTCH-709) JSParseFilter gets into an infinate loop and ets all the stack
Date Sun, 26 Apr 2009 18:42:30 GMT

    [ https://issues.apache.org/jira/browse/NUTCH-709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12702915#action_12702915
] 

Jeff Shafer edited comment on NUTCH-709 at 4/26/09 11:42 AM:
-------------------------------------------------------------

I can confirm that this patch works when applied to nutch-1.0 (release) running on hadoop
0.19.1.   I encountered the same stack overflow error with a custom max content length of
4MB.   Julien, I tried looking through the logs to find the URL that was causing the problem,
but did not see anything entries of interest.  (I fetch first, and then parse, so perhaps
I won't see the URL in that case? The only output I get from the parser is related to PDF
files)



      was (Author: shafer):
    I can confirm that this patch works when applied to nutch-1.0 (release) running on hadoop
0.19.1.   I encountered the same stack overflow error with a custom max content length of
4MB.   Tim, I tried looking through the logs to find the URL that was causing the problem,
but did not see anything entries of interest.  (I fetch first, and then parse, so perhaps
I won't see the URL in that case? The only output I get from the parser is related to PDF
files)


  
> JSParseFilter gets into an infinate loop and ets all the stack 
> ---------------------------------------------------------------
>
>                 Key: NUTCH-709
>                 URL: https://issues.apache.org/jira/browse/NUTCH-709
>             Project: Nutch
>          Issue Type: Bug
>          Components: fetcher
>    Affects Versions: 1.0.0
>         Environment: Hadoop 0.19.0 running nutch trunk 
>            Reporter: Tim Hawkins
>         Attachments: JSParseFilter.error.patch
>
>
> When crawling pages with seperate fetch and parse, I see processes die becuase of stack
overflow. 
> Output is generaly.
> java.lang.StackOverflowError
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:146)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> 	at org.apache.nutch.parse.js.JSParseFilter.walk(JSParseFilter.java:148)
> Inspection of the code shows that this is a recursive call to walk(.....) 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message