manifoldcf-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Conlon <>
Subject Pushing extra items into an index (outside normal crawl job)
Date Thu, 15 Aug 2013 16:17:39 GMT
Hi All,

I've been asked to consider adding items to an index outside normal repository crawl job processing
(e.g. to reduce the latency of a document being added to a repository and being available
in the index)

My initial thoughts on this are that this doesn't really fit in with the current ManifoldCF

With that in mind, I've come up with a couple of ideas (neither tested, nor thought through!)
that I'd like to run past the list to see whether they:

a)      Have the possibility of being reasonable

b)      Might be something that could be passed back into the ManifoldCF project (perhaps
as a contrib)

Idea one (probably the most work, but perhaps architecturally most clean):

1)      Introduce the idea of priority into ManifoldCF queues

2)      Add an extra "mcf" web service that allows queue injection

Idea two (easiest, if it works, but quite "hacky"):

1)      Add a web service that uses some "mcf" code to send documents directly to the output

2)      Obviously, this can't go through the ManifoldCF queues

3)      Relies upon a normal mcf job to tidy up any anomalies that might have occurred (deleting
and re-ingesting would be fine, I think)

How do these sound?  Are they worth thinking about?  Or indeed (better yet!), is there a better
way I haven't thought of...?


Electronic mail messages entering and leaving Arup  business
systems are scanned for acceptability of content and viruses

View raw message