lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Charlie Hull <>
Subject Re: External file fields
Date Fri, 02 Feb 2018 09:39:24 GMT
On 01/02/2018 18:55, Brian Yee wrote:
> Hello,
> I want to use external file field to store frequently changing inventory and price data.
I got a proof of concept working with a mock text file and this will suit my needs.
> What is the best way to keep this file updated in a fast way. Ideally I would like to
read changes from a Kafka queue and write to the file. But it seems like I would have to open
the whole file, read the whole file, find the line I want to change, and write the whole file
for every change. Is there a better way to do that? That approach seems like it would be difficult/slow
if the file is several million lines long.
> Also, once I come up with a way to update the file quickly, what is the best way to distribute
the file to all the different solrcloud nodes in the correct directory?
Another approach would be the XJoin plugin we wrote - if you wait a few 
days we should have an updated patch for Solr v6.5 and possibly v7. 
XJoin lets you filter/join/rank Solr results using an external data source.



Charlie Hull
Flax - Open Source Enterprise Search

tel/fax: +44 (0)8700 118334
mobile:  +44 (0)7767 825828

View raw message