lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erick Erickson <erickerick...@gmail.com>
Subject Re: SOLR Index in shared/Network folder
Date Mon, 30 Mar 2015 15:00:56 GMT
First examine whether you can reduce the amount of data you keep
around, field norms, stored fields, etc. Here's a place to start:
http://stackoverflow.com/questions/10080881/solr-index-size-reduction

I have heard of people doing what you suggest, but be _very_ careful
that you don't accidentally write to the index from the slave.

I always have to ask. You're talking about reducing disk space because
you have "100's of G" indexes. Is it really worth the engineering
effort to save the disk space? Terabyte hard drives are < $100. Buy
two for the cost of untangling a single corrupt index because someone
(and someone will) caused the slaves to write to the index rather than
the master.

FWIW,
Erick

On Sun, Mar 29, 2015 at 10:42 PM, abhi Abhishek <abhi26691@gmail.com> wrote:
> Hello,
>      Thanks for the suggestions. My aim is to reduce the disk space usage.
> I have 1 master with 2 slave configured, where slaves are used for
> searching and master ingests new data replicated to slaves, but as my index
> size is in 100's of GB we see 3x times space overhead. i would like to
> reduce this overhead, can you suggest something for this?
>
> Thanks in Advance
>
> Best Regards,
> Abhishek
>
> On Sat, Mar 28, 2015 at 12:13 AM, Erick Erickson <erickerickson@gmail.com>
> wrote:
>
>> To pile on: If you're talking about pointing two Solr instances at the
>> _same_ index, it doesn't matter whether you are on NFS or not, you'll
>> have all sorts of problems. And if this is a SolrCloud installation,
>> it's particularly hard to get right.
>>
>> Please do not do this unless you have a very good reason, and please
>> tell us what the reason is so we can perhaps suggest alternatives.
>>
>> Best,
>> Erick
>>
>> On Fri, Mar 27, 2015 at 8:08 AM, Walter Underwood <wunder@wunderwood.org>
>> wrote:
>> > Several years ago, I accidentally put Solr indexes on an NFS volume and
>> it was 100X slower.
>> >
>> > If you have enough RAM, query speed should be OK, but startup time
>> (loading indexes into file buffers) could be really long. Indexing could be
>> quite slow.
>> >
>> > wunder
>> > Walter Underwood
>> > wunder@wunderwood.org
>> > http://observer.wunderwood.org/  (my blog)
>> >
>> >
>> > On Mar 26, 2015, at 11:31 PM, Shawn Heisey <apache@elyograg.org> wrote:
>> >
>> >> On 3/27/2015 12:06 AM, abhi Abhishek wrote:
>> >>> Greetings,
>> >>>      I am trying to use a network shared location as my index
>> directory.
>> >>> are there any known problems in using a Network File System for
>> running a
>> >>> SOLR Instance?
>> >>
>> >> It is not recommended.  You will probably need to change the lockType,
>> >> ... the default "native" probably will not work, and you might need to
>> >> change it to "none" to get it working ... but that disables an important
>> >> safety mechanism that prevents index corruption.
>> >>
>> >> http://stackoverflow.com/questions/9599529/solr-over-nfs-problems
>> >>
>> >> Thanks,
>> >> Shawn
>> >>
>> >
>>

Mime
View raw message