From user-return-12028-apmail-hbase-user-archive=hbase.apache.org@hbase.apache.org Tue Aug 17 22:12:12 2010 Return-Path: Delivered-To: apmail-hbase-user-archive@www.apache.org Received: (qmail 30715 invoked from network); 17 Aug 2010 22:12:11 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 17 Aug 2010 22:12:11 -0000 Received: (qmail 91232 invoked by uid 500); 17 Aug 2010 22:12:10 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 91199 invoked by uid 500); 17 Aug 2010 22:12:10 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 91191 invoked by uid 500); 17 Aug 2010 22:12:10 -0000 Delivered-To: apmail-hadoop-hbase-user@hadoop.apache.org Received: (qmail 91188 invoked by uid 99); 17 Aug 2010 22:12:10 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Aug 2010 22:12:10 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=FREEMAIL_FROM,HTML_MESSAGE,SPF_PASS,T_TO_NO_BRKTS_FREEMAIL X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of gstathis@gmail.com designates 74.125.82.176 as permitted sender) Received: from [74.125.82.176] (HELO mail-wy0-f176.google.com) (74.125.82.176) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 17 Aug 2010 22:12:03 +0000 Received: by wyb35 with SMTP id 35so10187692wyb.35 for ; Tue, 17 Aug 2010 15:11:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:mime-version:received:in-reply-to :references:from:date:message-id:subject:to:content-type; bh=J96b9m2k+bRoAWHIz/OIGdDPUm9fOKw5+kxiOXOeJUU=; b=pzNmdNiKzac/iwOJaiysExEgW2LVSoBofBzcrcsHLlytYed0Qfr8XJ9/hyHZ4gahSK drMjJB5fO/ImXJeHp/XvyD3Kshbodpq3BlC6Qx3Egzrjp4mKHsJFPKvLKnlHHviXcTbG +vGTIqLvJpFKtTj4yZJ2ZGD1F59gpVBVuNTq0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :content-type; b=CIoowJpPAThmxq25uc2W+cQ1f8J3N2yNihO2q0eg0H0Vab6ZTLg02eQ+32M1NZSUAF pNkxi7XwCxLcAuAf+6fU6BnXsuaR0CXCLvWLWQjpzhFx2B8B3P2LuKBEOx0uKisIm0nM 6Rwe7KAH+aW/AX/4oa1rGT9MixSzbMOuLFtnI= Received: by 10.216.157.72 with SMTP id n50mr1301209wek.4.1282083102228; Tue, 17 Aug 2010 15:11:42 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.72.207 with HTTP; Tue, 17 Aug 2010 15:11:22 -0700 (PDT) In-Reply-To: References: From: George Stathis Date: Tue, 17 Aug 2010 18:11:22 -0400 Message-ID: Subject: Re: High OS Load Numbers when idle To: hbase-user@hadoop.apache.org Content-Type: multipart/alternative; boundary=0016e6546d9ee18341048e0c3bc0 --0016e6546d9ee18341048e0c3bc0 Content-Type: text/plain; charset=ISO-8859-1 No takers? :-) Am I missing something too obvious? On Tue, Aug 17, 2010 at 2:03 PM, George Stathis wrote: > Hello, > > We have just setup a new cluster on EC2 using Hadoop 0.20.2 and HBase > 0.20.3. Our small setup as of right now consists of one master and four > slaves with a replication factor of 2: > > Master: xLarge instance with 2 CPUs and 17.5 GB RAM - runs 1 namenode, 1 > secondarynamenode, 1 jobtracker, 1 hbasemaster, 1 zookeeper (uses its' own > dedicated EMS drive) > Slaves: xLarge instance with 2 CPUs and 17.5 GB RAM each - run 1 datanode, > 1 tasktracker, 1 regionserver > > We have also installed Ganglia to monitor the cluster stats as we are about > to run some performance tests but, right out of the box, we are noticing > high system loads (especially on the master node) without any activity > happening on the clister. Of course, the CPUs are not being utilized at all, > but Ganglia is reporting almost all nodes in the red as the 1, 5 an 15 > minute load times are all above 100% most of the time (i.e. there are more > than two processes at a time competing for the 2 CPUs time). > > Question1: is this normal? > Question2: does it matter since each process barely uses any of the CPU > time? > > Thank you in advance and pardon the noob questions. > > -GS > --0016e6546d9ee18341048e0c3bc0--