lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vadim Kisselmann <v.kisselm...@googlemail.com>
Subject Re: Still too many files after running solr optimization
Date Wed, 28 Sep 2011 15:03:10 GMT
we had an understanding problem:)

docs are the docs in index.
files are the files in the index directory (index parts).

during the optimization you don't delete docs if they are don't flagged as
deleted.
but you merge your index und delete the files in your index directory, thats
right.

after an second optimize the files are deleted which were opened for
reading.

Regards



2011/9/28 Manish Bafna <manish.bafna.82@gmail.com>

> We tested it so many times.
> 1st time we optimize, the new index file is created (merged one), but
> the existing index files are not deleted (because they might be still
> open for reading)
> 2nd time optimize, other than the new index file, all else gets deleted.
>
> This is happening specifically on Windows.
>
> On Wed, Sep 28, 2011 at 8:23 PM, Vadim Kisselmann
> <v.kisselmann@googlemail.com> wrote:
> > 2011/9/28 Manish Bafna <manish.bafna.82@gmail.com>
> >
> >> >>Will it not merge the index?
> >>
> >
> > yes
> >
> >
> >> >>While merging on windows, the old index files dont get deleted.
> >> >>(Windows has an issue where the file opened for reading cannot be
> >> >>deleted)
> >> >>
> >> >>So, if you call optimize again, it will delete the older index files.
> >>
> >> no.
> > during optimize you only delete docs, which are flagged as deleted. no
> > matter how old they are.
> > if your numDocs and maxDocs have the same number of Docs, you only
> rebuild
> > and merge your index, but you delete nothing.
> >
> > Regards
> >
> >
> >
> >
> >> On Wed, Sep 28, 2011 at 6:43 PM, Vadim Kisselmann
> >> <v.kisselmann@googlemail.com> wrote:
> >> > if numDocs und maxDocs have the same mumber of docs nothing will be
> >> deleted
> >> > on optimize.
> >> > You only rebuild your index.
> >> >
> >> > Regards
> >> > Vadim
> >> >
> >> >
> >> >
> >> >
> >> > 2011/9/28 Kissue Kissue <kissuenow@gmail.com>
> >> >
> >> >> numDocs and maxDocs are same size.
> >> >>
> >> >> I was worried because when i used to use only Lucene for the same
> >> indexing,
> >> >> before optimization there are many files but after optimization i
> always
> >> >> end
> >> >> up with just 3 files in my index filder. Just want to find out if
> this
> >> was
> >> >> ok.
> >> >>
> >> >> Thanks
> >> >>
> >> >> On Wed, Sep 28, 2011 at 1:23 PM, Vadim Kisselmann <
> >> >> v.kisselmann@googlemail.com> wrote:
> >> >>
> >> >> > why should the optimization reduce the number of files?
> >> >> > It happens only when you indexing docs with same unique key.
> >> >> >
> >> >> > Have you differences in numDocs und maxDocs after optimize?
> >> >> > If yes:
> >> >> > how is your optimize command ?
> >> >> >
> >> >> > Regards
> >> >> > Vadim
> >> >> >
> >> >> >
> >> >> >
> >> >> > 2011/9/28 Manish Bafna <manish.bafna.82@gmail.com>
> >> >> >
> >> >> > > Try to do optimize twice.
> >> >> > > The 2nd one will be quick and will delete lot of files.
> >> >> > >
> >> >> > > On Wed, Sep 28, 2011 at 5:26 PM, Kissue Kissue <
> kissuenow@gmail.com
> >> >
> >> >> > > wrote:
> >> >> > > > Hi,
> >> >> > > >
> >> >> > > > I am using solr 3.3. I noticed  that after indexing
about 700,
> 000
> >> >> > > records
> >> >> > > > and running optimization at the end, i still have about
91
> files
> >> in
> >> >> my
> >> >> > > index
> >> >> > > > directory. I thought that optimization was supposed
to reduce
> the
> >> >> > number
> >> >> > > of
> >> >> > > > files.
> >> >> > > >
> >> >> > > > My settings are the default that came with Solr (mergefactor,
> etc)
> >> >> > > >
> >> >> > > > Any ideas what i could be doing wrong?
> >> >> > > >
> >> >> > >
> >> >> >
> >> >>
> >> >
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message