lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Lu" <>
Subject Re: Lucene vs. Database
Date Wed, 01 Oct 2008 08:08:25 GMT
Since you have a lot of data, it wouldn't be wise to put those extra data
into Lucene index for storage. You will need to keep the data in sync, and
update those extra data in lucene index unnecessarily.
Lucene index is just like B-Tree index in database. It's just an auxiliary
data structure, and it'll consume resources to maintain the data structure.

So basically, using Lucene index as a storage would depend on how often you
data are updated.

The cut-off line usually is to put data that you will need to render search
results into Lucene index. And after user click on one of the search result,
just use database later on. But this isn't always correct. When you have 10
result per page, selecting the details from the database based on ids may
not be that costly at all.

Chris Lu
Instant Scalable Full-Text Search On Any Database/Application
Lucene Database Search in 3 minutes:
DBSight customer, a shopping comparison site, (anonymous per request) got
2.6 Million Euro funding!

On Wed, Oct 1, 2008 at 12:43 AM, agatone <> wrote:

> Hi,
> I asked this question already on "lucene-general" list but also got advised
> to ask here too.
> I'm working on a project that has big database in the background (some
> tables have about 1500000 rows). We decided to use Lucene for "faster"
> search. Our search works similar as all searches: you write search string,
> get list of hits with detail link. But there is dilemma if we should store
> more data into index than it's needed.
> One side of developing team insists that we should use lucene index as
> somekind of storage for data so when you get hit, you go onto details and
> then again use lucene to find document that matches the selected ID and
> take
> the data from Lucene index. So in the end you end with copying complete
> database tables into the lucene index.
> Other side insists on storing to index only data that is displayed directly
> to the user when showing the search results list and needed for search
> criteria. When you go onto details, you have the matching ID so you can
> pickup that row from database by that ID rather than search it inside
> Lucene
> index.
> Can someone please describe drawbacks and advantages of both approaches.
> Actually can someone write down what's the actual profit, where and when of
> the Lucene itself in real production env.
> IT would be great if there is anyone who could write his experience with
> indexing and searching large amount of data.
> Thank you
> --
> View this message in context:
> Sent from the Lucene - Java Users mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message