lucene-solr-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santanu8939967892 <mishra.sant...@gmail.com>
Subject Re: DIH to index the data - 250 millions - Need a best architecture
Date Tue, 30 Jul 2013 06:23:00 GMT
Hi Shawn,
     Yes, your assumption is correct. The index size is around 250 GB and
we index 20/30 meta data and store around 50.
     We have plan for a Solr cloud architecture having two nodes one Master
and other one is replica of the master (replication factor 2) with multiple
zookeeper ensemble. We will have multiple shards for each Master and
replica node.
Is above architecture a fit for production deployment for an improved index
and query performance.
Do we require 64 GB RAM or less will work for us.

With Regards,
Santanu



On Tue, Jul 30, 2013 at 12:59 AM, Mikhail Khludnev <
mkhludnev@griddynamics.com> wrote:

> Mishra,
> What if you setup DIH with single SQLEntityProcessor without caching, does
> it works for you?
>
>
> On Mon, Jul 29, 2013 at 4:00 PM, Santanu8939967892 <
> mishra.santanu@gmail.com
> > wrote:
>
> > Hi,
> >    I have a huge volume of DB records, which is close to 250 millions.
> > I am going to use DIH to index the data into Solr.
> > I need a best architecture to index and query the data in an efficient
> > manner.
> > I am using windows server 2008 with 16 GB RAM, zion processor and Solr
> 4.4.
> >
> >
> > With Regards,
> > Santanu
> >
>
>
>
> --
> Sincerely yours
> Mikhail Khludnev
> Principal Engineer,
> Grid Dynamics
>
> <http://www.griddynamics.com>
>  <mkhludnev@griddynamics.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message