lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Walter Underwood <wun...@wunderwood.org>
Subject Re: SOLR Index in shared/Network folder
Date Mon, 30 Mar 2015 15:13:11 GMT
I suggest that you do not try to save money on disk space. Disk is cheap.

You will spend weeks of expensive engineering time trying to make this work. Once you make
it work, it will be slow an unreliable.

300GB Amazon EBS volumes are $180/year, $360/year for SSD. Just spend the money.

wunder
Walter Underwood
wunder@wunderwood.org
http://observer.wunderwood.org/  (my blog)


On Mar 29, 2015, at 10:42 PM, abhi Abhishek <abhi26691@gmail.com> wrote:

> Hello,
>     Thanks for the suggestions. My aim is to reduce the disk space usage.
> I have 1 master with 2 slave configured, where slaves are used for
> searching and master ingests new data replicated to slaves, but as my index
> size is in 100's of GB we see 3x times space overhead. i would like to
> reduce this overhead, can you suggest something for this?
> 
> Thanks in Advance
> 
> Best Regards,
> Abhishek
> 
> On Sat, Mar 28, 2015 at 12:13 AM, Erick Erickson <erickerickson@gmail.com>
> wrote:
> 
>> To pile on: If you're talking about pointing two Solr instances at the
>> _same_ index, it doesn't matter whether you are on NFS or not, you'll
>> have all sorts of problems. And if this is a SolrCloud installation,
>> it's particularly hard to get right.
>> 
>> Please do not do this unless you have a very good reason, and please
>> tell us what the reason is so we can perhaps suggest alternatives.
>> 
>> Best,
>> Erick
>> 
>> On Fri, Mar 27, 2015 at 8:08 AM, Walter Underwood <wunder@wunderwood.org>
>> wrote:
>>> Several years ago, I accidentally put Solr indexes on an NFS volume and
>> it was 100X slower.
>>> 
>>> If you have enough RAM, query speed should be OK, but startup time
>> (loading indexes into file buffers) could be really long. Indexing could be
>> quite slow.
>>> 
>>> wunder
>>> Walter Underwood
>>> wunder@wunderwood.org
>>> http://observer.wunderwood.org/  (my blog)
>>> 
>>> 
>>> On Mar 26, 2015, at 11:31 PM, Shawn Heisey <apache@elyograg.org> wrote:
>>> 
>>>> On 3/27/2015 12:06 AM, abhi Abhishek wrote:
>>>>> Greetings,
>>>>>     I am trying to use a network shared location as my index
>> directory.
>>>>> are there any known problems in using a Network File System for
>> running a
>>>>> SOLR Instance?
>>>> 
>>>> It is not recommended.  You will probably need to change the lockType,
>>>> ... the default "native" probably will not work, and you might need to
>>>> change it to "none" to get it working ... but that disables an important
>>>> safety mechanism that prevents index corruption.
>>>> 
>>>> http://stackoverflow.com/questions/9599529/solr-over-nfs-problems
>>>> 
>>>> Thanks,
>>>> Shawn
>>>> 
>>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message