hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shubham Sharma <ssha...@pivotal.io>
Subject Re: HAWQ-1548 - Ambiguous message while logging hawq utilization
Date Tue, 14 Nov 2017 04:10:49 GMT
Hi Wen,

I thought the same but when I look at - resource_manager_respool_status
using - SELECT * FROM dump_resource_manager_status(2); All segments are
registered and each has configuration 98304MB and 12.000000 CORE

On Mon, Nov 13, 2017 at 7:40 PM, Wen Lin <wlin@pivotal.io> wrote:

> Hi, Shubham,
>
> The second message means that, this YARN cluster's total resource is
> 1376256
> MB, 168.000000 CORE.
> And HAWQ clusters have the resource: 98304MB, 12.000000 CORE of the whole
> YARN cluster now.
> The other resources in YARN haven't been allocated for HAWQ.
> Maybe there are no HAWQ segments running on these segments at the moment,
> or other reasons.
>
> Thanks!
>
>
> On Tue, Nov 14, 2017 at 9:25 AM, Shubham Sharma <ssharma@pivotal.io>
> wrote:
>
> > While YARN mode is enabled, resource broker logs two things -
> >
> >    - YARN cluster total resource
> >    - HAWQ's total resource per node.
> >
> > Following messages are logged
> >
> > 2017-11-11 23:21:40.944904
> > UTC,,,p549330,th900077856,,,,0,con4,,seg-10000,,,,,"LOG","
> 00000","Resource
> > manager YARN resource broker counted YARN cluster having total
> > resource (1376256 MB, 168.000000
> > CORE).",,,,,,,0,,"resourcebroker_LIBYARN.c",776,
> >
> > 2017-11-11 23:21:40.944921
> > UTC,,,p549330,th900077856,,,,0,con4,,seg-10000,,,,,"LOG","
> 00000","Resource
> > manager YARN resource broker counted HAWQ cluster now having (98304
> > MB, 12.000000 CORE) in a YARN cluster of total resource (1376256 MB,
> > 168.000000 CORE).",,,,,,,0,,"resourcebroker_LIBYARN.c",785,
> >
> > The second message shown above is ambiguous, After reading the sentence
> > below it looks like that complete Hawq cluster in whole has only 98304 MB
> > and 12 cores. However according to the configuration it should be 98304
> MB
> > and 12 cores per segment server.
> >
> > Resource manager YARN resource broker counted HAWQ cluster now having
> > (98304 MB, 12.000000 CORE) in a YARN cluster of total resource
> > (1376256 MB, 168.000000 CORE).
> >
> > Either the wrong variables are printed or we can correct the message to
> > represent that the resources logged are per node. As this can confuse the
> > user into thinking that hawq cluster does not have enough resources.
> >
> > Is this understanding correct ?
> >
> > Regards,
> > Shubham Sharma
> >
>



-- 
Regards,
Shubham Sharma
Staff Customer Engineer
Pivotal Global Support Services
ssharma@pivotal.io
Direct Tel: +1(510)-304-8201
Office Hours: Mon-Fri 9:00 am to 5:00 pm PDT
Out of Office Hours Contact +1 877-477-2269

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message