lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Clemens Marschner" <>
Subject Fw: LARM / Re: Avalonized WebCrawler
Date Fri, 31 Jan 2003 12:46:33 GMT
While sipping a grande latte, I came up with the following questions (more
tom come):

1. I wonder how ...crawl.fetcher is working, since there seem to be some
  - DefaultFetcherTaskFacotry.xinfo (o<->t)
    contains a reference to
    which doesn't exist

2. Why crawl.Main and crawl.CrawlMain?

3. Do you think dynamically configuring the whole pipeline from a config
file would be possible? The contents of com.celavi.crawl.Main.service()
should come from a config file, say pipeline_xy.xml (more than one pipeline
config should be possible, say crawler_pipeline and indexer_pipeline).
Depending on the contents of this file, another config file should contain
the config values for each component (say crawl_full, crawl_incrementally)

4. What is your rule of thumb what becomes a component and what stays a

5. Why Fortress and not a different container (just curious, I don't have
any preference)?

6. It appears to me that Fortress is creating proxy components that act as
facades to the underlying component interfaces (am I right here?). This is
exactly what I wanted to avoid. It simply becomes too heavy weighted (unless
we use typical component patterns). Since we may well create 100,000
URLMessages per second, it would kill us to send every call to
urlMessageFactory.createURLMessage through a proxy. I wonder if the other
available containers work the same way? (I know Phoenix doesn't do this)

By the way, I compiled it with Eclipse (first time...) A walk in the park...

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message