lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Glen Newton <glen.new...@gmail.com>
Subject Re: How to reduce the Solr index size..
Date Thu, 27 Aug 2009 20:35:24 GMT
2009/8/27 Fuad Efendi <fuad@efendi.ca>:
> stored="true" means that this piece of info will be stored in a filesystem.
> So that your index will contain 1Mb of pure log PLUS some info related to
> indexing itself: terms, etc.
>
> Search speed is more important than index size...

Not if you run out of space for the index. :-)

> And note this: message field contains actual log, stored="true", so that
> only this field will make 1Mb if not indexed
>
>
> -----Original Message-----
> From: Silent Surfer [mailto:silentsurfer77@yahoo.com]
> Sent: August-20-09 11:01 AM
> To: Solr User
> Subject: How to reduce the Solr index size..
>
> Hi,
>
> I am newbie to Solr. We recently started using Solr.
>
> We are using Solr to process the server logs. We are creating the indexes
> for each line of the logs, so that users would be able to do a fine grain
> search upto second/ms.
>
> Now what we are observing is , the index size that is being created is
> almost double the size of the actual log size. i.e if the logs size is say 1
> MB, the actual index size is around 2 MB.
>
> Could anyone let us know what can be done to reduce the index size. Do we
> need to change any configurations/delete any files which are created during
> the indexing processes, but not required for searching..
>
> Our schema is as follows:
>
>   <field name="pkey" type="string" indexed="true" stored="true"
> required="false" />
>   <field name="date" type="date" indexed="true" stored="true"
> omitNorms="true"/>
>   <field name="level" type="string" indexed="true" stored="true"/>
>   <field name="app" type="string" indexed="true" stored="true"/>
>   <field name="server" type="string" indexed="true" stored="true"/>
>   <field name="port" type="string" indexed="true" stored="true"/>
>   <field name="class" type="string" indexed="true" stored="true"/>
>   <field name="method" type="string" indexed="true" stored="true"/>
>   <field name="filename" type="string" indexed="true" stored="true"/>
>   <field name="linenumber" type="string" indexed="true" stored="true"/>
>   <field name="message" type="text" indexed="true" stored="true"/>
>
> message field holds the actual logtext.
>
> Thanks,
> sS
>
>
>
>
>
>
>



-- 

-

Mime
View raw message