nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdy Galema (Commented) (JIRA)" <>
Subject [jira] [Commented] (NUTCH-1302) nutchgora job failures should be noticed by submitter
Date Tue, 06 Mar 2012 13:16:59 GMT


Ferdy Galema commented on NUTCH-1302:

NutchJob is a nice wrapper of Hadoop's Job, so that I could override Job.waitForCompletion(verbose)
in order to throw an exception whenever a job fails. (A job fails typically whenever one or
more tasks have failed repeatedly.)

Because some people may be reliant upon silent job failures (could not think of why, but just
in case), I created an expert option that disables this behaviour, namely "fail.on.job.failure".
When this property is set to false, jobs will fail silently like they did before.
> nutchgora job failures should be noticed by submitter
> -----------------------------------------------------
>                 Key: NUTCH-1302
>                 URL:
>             Project: Nutch
>          Issue Type: Bug
>            Reporter: Ferdy Galema
>            Assignee: Ferdy Galema
> I stumbled upon an issue where crawling seem to go right, only to notice much later on
that jobs actually failed as a whole.
> This is caused because for most jobs that are submitted, Nutchgora does not check the
'succeeded' boolean that is returned. This should be done and acted upon appropriately. (Either
throwing an exception or returning non-zero exit codes).

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


View raw message