lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Feinstein" <>
Subject LogFormatter
Date Tue, 03 Jan 2006 10:42:52 GMT
This is my first email to the community. In our ( project we integrated multiple
lucene indexes and a nutch
package. We use the packages for about two years. Lucene is working great and we could use
it as a "black box".
Unfortunately, we had to modify Nutch package for our usage because of some leaks and a design
problem in LogFormatter
class. This fact force us to use a customized version of Nutch and avoid to update versions.
At the last week we decided
to move to the latest version of Nutch but discovered that the LogFormatter still have the
same problem (leaks I believe
has been fixed).
The problem:
In Nutch project LogFormatter has duplicated functionality:
1) Logger records format and
2) Severe error handler
The first usage is standard and usually could be overwritten by a user of the package by modifying
The second usage is much more problematic because it affects the behavior of the whole application
(not only Nutch
package). To support the error handling LogFormatter enforce usage of the formatter class
by all classes of the whole
application which uses Nutch package. This is done by overwriting all the system handlers
java.util.logging.Handler). This operation prevents the application to use its own log formatter.
Also this cause
LogFormatter.hasLoggedSevere() to be sensitive to all severe records in the big system but
not only to relevant. More
than that this flag, LogFormatter.loggedSevere is never cleaned what means if an application
had one, even unrelated
severe record, tools like Fetcher will never run until the application will be restarted.
I would like to suggest the following solutions:
1) To separate the functionality of log formatting and error handling or
2) Change LogFormatter class to be affected only by nutch package functions
For my opinion the first solution is much better especially if error handling will be encapsulated
for each task. I have
found the following usages of LogFormatter.hasLoggedSevere():
- Fetcher
- URLFilterChecker
- ParseSegment
Unfortunately I'm not familiar enough with the usages above to implement this solution that
why I suggest the second
one. The attached file is our implementation of LogFormatter class which we use for more than
a year. I hope this change
will be accepted by the community.
Daniel Feinstein
Lead Software Engineer <> 


This email may contain confidential and privileged material for the sole use of the intended
recipient. Any review or
distribution by others is strictly prohibited. If you are not the intended recipient please
contact the sender and
delete all copies.


View raw message