hawq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kyle Roberts <krobe...@pivotal.io>
Subject Default LOG level for "data locality ratio:" events contribute to Hawq Master log bloat
Date Thu, 30 Nov 2017 21:41:50 GMT
Issue:


Default LOG level for "data locality ratio:" events contribute to Hawq
Master log bloat.  Should only be needed when investigating/tuning
performance.

Case Scenario:

*"data locality ratio:"* events can report multiple times and, depending
upon the query, many times.


Sample event:


2017-11-30 20:33:39.857251
GMT,"gpadmin","testdb",p280349,th-909919968,"[local]",,2017-11-30 20:06:35
GMT,1630750,con25419,cmd28,seg-1,,,,sx2,"LOG","00000","data locality ratio:
1.000; virtual segment number: 1; different host number: 1; virtual segment
number per host(avg/min/max): (1/1/1); segment size(avg/min/max): (1016.000
B/1016 B/1016 B); segment size with penalty(avg/min/max): (1016.000 B/1016
B/1016 B); continuity(avg/min/max): (1.000/1.000/1.000).",,,,,"SQL
statement ""select count(*)::float4 from public.test_tbl as Ta where Ta.id
is null""","analyze test_tbl;",0,,"cdbdatalocality.c",3372,



For example, Hawq master logs are seen with close to *20% of the file size*
is from only the *"data locality ratio:" *events.


Product Enhancement suggestion:


Because these are only needed for performance tuning (or for investigation):


In:


https://github.com/apache/incubator-hawq/blob/master/src/backend/cdb/cdbdatalocality.c#L3372


Could we change default LOG level to something like DEBUG1, etc?




- Kyle

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message