nutch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rod Taylor <...@sitesell.com>
Subject Should nutch try to reduce first?
Date Fri, 09 Dec 2005 04:57:35 GMT
When you run multiple commands within nutch it seems to process the
pending tasks in the order that they were added to the queue.  In some
cases this means you may be 50% through many jobs (complete map but not
reduce) while processes maps for yet more jobs.

I think Nutch should prioritize a pending reduce before a pending map as
it keeps things going through (other processes may depend on the
results) and allows temporary diskspace to be freed.
-- 
Rod Taylor <rbt@sitesell.com>


Mime
View raw message