lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rajini maski <rajinima...@gmail.com>
Subject Re: OutOfMemoryErrors
Date Wed, 18 Aug 2010 03:54:49 GMT
Yeah fine..I will do that...Before the merge Factor was 10 itself ...After
finding this error I just set its value higher assuming if that could be
error anyway... Will re change it..

The ramBufferSize  is 256MB... Do I need to  change this value to higher?


On Wed, Aug 18, 2010 at 12:27 AM, Jay Hill <jayallenhill@gmail.com> wrote:

> A merge factor of 100 is very high and out of the norm. Try starting with a
> value of 10. I've never seen a running system with a value anywhere near
> this high.
>
> Also, what is your setting for ramBufferSizeMB?
>
> -Jay
>
> On Tue, Aug 17, 2010 at 10:46 AM, rajini maski <rajinimaski@gmail.com
> >wrote:
>
> > yeah sorry I forgot to mention others...
> >
> > <mergeFactor>100</mergeFactor>
> > <maxBufferedDocs>1000</maxBufferedDocs>
> > <maxMergeDocs>100000</maxMergeDocs>
> > <maxFieldLength>10000</maxFieldLength>
> >
> > above are the values....
> >
> > Is this because of values here...initially I had mergeFactor parameter
> -10
> > and maxMergedocs-10000....With the same error i changed them to above
> > values..Yet I got that error after index was about 2lacs docs...
> >
> > On Tue, Aug 17, 2010 at 11:04 PM, Erick Erickson <
> erickerickson@gmail.com
> > >wrote:
> >
> > > There are more merge paramaters, what values do you have for these:
> > >
> > > <mergeFactor>10</mergeFactor>
> > > <maxBufferedDocs>1000</maxBufferedDocs>
> > > <maxMergeDocs>2147483647</maxMergeDocs>
> > > <maxFieldLength>10000</maxFieldLength>
> > >
> > > See: http://wiki.apache.org/solr/SolrConfigXml
> > >
> > > Hope that formatting comes through the various mail programs OK....
> > >
> > > Also, what else happens while you're indexing? Do you search
> > > while indexing? How often do you commit your changes?
> > >
> > >
> > >
> > > On Tue, Aug 17, 2010 at 1:18 PM, rajini maski <rajinimaski@gmail.com>
> > > wrote:
> > >
> > > > <mergefactor>100 </mergefactor>
> > > > JVM Initial memory pool -256MB
> > > >       Maximum memory pool -1024MB
> > > >
> > > > <add>
> > > > <doc>
> > > > <field>long:ID</field>
> > > > <field>str:Body</field>
> > > > ....
> > > > ....12 fields
> > > > </filed>
> > > > </doc>
> > > > </add>
> > > > I have a solr instance in solr folder (D:/Solr) free space in disc is
> > > > 24.3GB
> > > > .. How will I get to know what portion of memory is solr using ?
> > > >
> > > >
> > > >
> > > > On Tue, Aug 17, 2010 at 10:11 PM, Erick Erickson <
> > > erickerickson@gmail.com
> > > > >wrote:
> > > >
> > > > > You shouldn't be getting this error at all unless you're doing
> > > something
> > > > > out of the ordinary. So, it'd help if you told us:
> > > > >
> > > > > >What parameters you have set for merging
> > > > > >What parameters you have set for the JVM
> > > > > >What kind of documents are you indexing?
> > > > >
> > > > > The memory you have is irrelevant if you only allocate a small
> > > > > portion of it for the running process...
> > > > >
> > > > > Best
> > > > > Erick
> > > > >
> > > > > On Tue, Aug 17, 2010 at 7:35 AM, rajini maski <
> rajinimaski@gmail.com
> > >
> > > > > wrote:
> > > > >
> > > > > > I am getting it while indexing data to solr not while querying...
> > > > > > Though I have enough memory space upto 40GB and I my indexing
> data
> > is
> > > > > just
> > > > > > 5-6 GB yet that particular error is seldom observed... (SEVERE
> > ERROR
> > > :
> > > > > JAVA
> > > > > > HEAP SPACE , OUT OF MEMORY ERROR )
> > > > > > I could see one lock file generated in the data/index path just
> > after
> > > > > this
> > > > > > error.
> > > > > >
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 17, 2010 at 4:49 PM, Peter Karich <peathal@yahoo.de>
> > > > wrote:
> > > > > >
> > > > > > >
> > > > > > > > Is there a way to verify that I have added correctlly?
> > > > > > > >
> > > > > > >
> > > > > > > on linux you can do
> > > > > > > ps -elf | grep Boot
> > > > > > > and see if the java command has the parameters added.
> > > > > > >
> > > > > > > @all: why and when do you get those OOMs? while querying?
which
> > > > queries
> > > > > > > in detail?
> > > > > > >
> > > > > > > Regards,
> > > > > > > Peter.
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message