hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Imran M Yousuf <imyou...@gmail.com>
Subject Re: About test/production server configuration
Date Wed, 07 Apr 2010 01:39:28 GMT
Hi Jonathan,

Thanks for your reply. Please find my replies inline.

On Wed, Apr 7, 2010 at 4:04 AM, Jonathan Gray <jgray@facebook.com> wrote:
> Or if you have a budget in mind, we can help you determine what would be the best way
to allocate those dollars.

That would be just great. Budget provisioned for the whole system is
approximately 27,000 USD. Among that we have budgeted for the
Hadoop+HBase cluster to 13,500 USD (for 10 servers).

>> <snip />
>> Have you run Solr atop HDFS?  I doubt this will be performant.

We haven't tested it yet, but it is in our scope; after testing it we
will decide on which path to take.

>> Also, to properly scope your cluster, you need to come up with actual
>> number targets if you want to be able to accurately provision hardware.
>> "not much" data now, but "lots" of data later could mean anything.
>> Decide what you want to provision for and then you can accurately do
>> so.

Hmm, I am not sure I understand correctly about provisioning but I am
giving it a try.
Our system composes of web applications for a CMS,
Accounting+Inventory System (SaaS), another web application
integrating CMS and Accounts, and Solr as a search engine. So in times
of data for the setup I would like to support 6TB of data and 4
Billion rows. Some details are as follows.

2000 organizations using the SaaS. Each with 500 inventory items. Each
inventory item with MM would be at least 300k at an average. We want
to support a million transactions per organizations (average) with
each being 2k at an average. So total for the accounting system is
4,300 GB ~ 5 TB (approx.) of data and 2,001,002,000 rows minimum.

Each inventory item will be a content in the CMS; in addition users
can organize their contents; plus the MM will also be a content
(basically a copy of the record mentioned above). 750 GB and 2M rows

I hope this helps. Eagerly waiting for some direction :).

Thank you,


> <snip />

Imran M Yousuf
Entrepreneur & Software Engineer
Smart IT Engineering
Dhaka, Bangladesh
Email: imran@smartitengineering.com
Blog: http://imyousuf-tech.blogs.smartitengineering.com/
Mobile: +880-1711402557

View raw message