hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean (Contractor)" <dean.hil...@broadridge.com>
Subject RE: can I do this to get a sample benchmark...
Date Thu, 09 Dec 2010 15:59:54 GMT
Ok, I finally answered one of my questions when running the rebalance
tool.  One node is 180gig and another 20gig so it technically in balance
with a 10% ratio.  When I ran 1%, it did rebalance.  I guess I expected
hbase to be writing to both nodes but was only writing on one.  I need
to get to a more equalized cluster first to run more tests.  

 

QUICK QUESTION: Is there a quick way to limit a node to the size of data
it will use(to lower that 180gig number to 20)(or do I just have to
create a new linux partition with new mount point and point that node
there)?

 

Thanks,

Dean

 

From: Hiller, Dean (Contractor) 
Sent: Wednesday, December 08, 2010 5:57 PM
To: 'hbase-user@hadoop.apache.org'
Subject: can I do this to get a sample benchmark...

 

I would really just like to insert 1,000,000 rows on 2 machines like so

 

[root@localhost hadoop]# ./bin/hadoop
org.apache.hadoop.hbase.PerformanceEvalua

tion --rows=1000000 sequentialWrite 2

 

and then insert 1,000,000 rows on 3 machines

 

[root@localhost hadoop]# ./bin/hadoop
org.apache.hadoop.hbase.PerformanceEvalua

tion --rows=1000000 sequentialWrite 3

 

but when I ran the first test and then looked at the size afterwards, I
had 3.6 gigs on one

node and 0 gigs on the other node.  I have dfs.replication set to 1(ie.
I don't want it for

this test right now).  I am just trying to play with the system a bit
here but not sure why

it is very imbalanced on the writes???  Is there a block size I should
change or something?

 

Once I get done with this, I want to write a map job to take data from
another db and stuff

It into my small little cluster to test it all out as a small prototype.

 

Thanks,

Dean


This message and any attachments are intended only for the use of the addressee and
may contain information that is privileged and confidential. If the reader of the 
message is not the intended recipient or an authorized representative of the
intended recipient, you are hereby notified that any dissemination of this
communication is strictly prohibited. If you have received this communication in
error, please notify us immediately by e-mail and delete the message and any
attachments from your system.

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message