lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alice" <aliceli...@gmail.com>
Subject RE: Huge Index
Date Thu, 11 Jan 2007 17:19:29 GMT
You're right. The computer doesn't crash but slows to death.

I'm developing in my workstation and the application will run in a server, a
better machine but not that great.

The indexing will sure run at night no matter the performance but it is not
desired to take many hours.

-----Original Message-----
From: Ruslan Sivak [mailto:rsivak@istandfor.com] 
Sent: quinta-feira, 11 de janeiro de 2007 15:12
To: java-user@lucene.apache.org
Subject: Re: Huge Index

Alice,

If you have a computer that crashes once you put a lot of load on it, 
I'd say you have bigger problems then the speed of the indexing.  A 
computer should not crash, no matter how much load you put on it.  If 
you have such a huge database, I can't believe that you don't have 
access to other computers.  You can even use workstations and run the 
indexing at night, and then merge it to a central repository. 

Lucene does allow for index merging, right?

Russ



Alice wrote:
> Unfortunately I can't use multiple machines.
>
> And I cannot start lots of threads because the server crashes.
>
> -----Original Message-----
> From: Russ [mailto:rsivak@istandfor.com] 
> Sent: quinta-feira, 11 de janeiro de 2007 14:33
> To: java-user@lucene.apache.org
> Subject: Re: Huge Index
>
> Can you use multiple threads/machines to index the data into separate
> indexes, and then combine them?
>
> Russ
> Sent wirelessly via BlackBerry from T-Mobile.  
>
> -----Original Message-----
> From: "Alice" <alicelista@gmail.com>
> Date: Thu, 11 Jan 2007 13:47:36 
> To:<java-user@lucene.apache.org>
> Subject: Huge Index
>
> Hello!
>
>  
>
> I have to index 37million documents retrieved from the database.
>
>  
>
> I was trying to do by loading intervals of 10000 records but it is too
slow.
>
>  
>
> Anybody could sugest a better way to get all the data indexed in a
> reasonable time?
>
>  
>
> Thanks
>
>  
>
>  
>
>  
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>   


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message