kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jon Yeargers <jon.yearg...@cedexis.com>
Subject Re: spontaneous / unwanted exits from KStream apps
Date Wed, 21 Dec 2016 13:02:57 GMT
Did I say memory usage was stable? Lies.

After ~15min it's up to 50% - and climbing.

20 min: 63%

Eventually the OS killed it. Didn't generate a log this time though. Found
this snip in /var/log/messages:

Dec 21 12:56:19 ip-172-16-101-108 kernel: [2901342.207241] java invoked
oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0


Some stack dump bits as well. This bug (or one v much like it) was fixed in
the 0.10.2.0-SNAPSHOT I was previously using. The unhappy build comes from
0.10.1.1-RC1 and trunk.



On Wed, Dec 21, 2016 at 4:32 AM, Jon Yeargers <jon.yeargers@cedexis.com>
wrote:

> Found this treasure lurking in my app folder:
>
> Shows that the process was OOM-killed by the OS. Have restarted to see if
> it will reproduce - so far memory usage seems stable.
>
>
>
> On Wed, Dec 21, 2016 at 3:05 AM, Jon Yeargers <jon.yeargers@cedexis.com>
> wrote:
>
>> I upgraded my app(s) to 0.10.1.1-rc1 but I'm still seeing these
>> error-free shutdowns. Something is taking my app down after varying lengths
>> of time (10 minutes to several hours). Doesn't matter if Im running one or
>> many instances.
>>
>> Suggestions on where to look? I've sent several debug logs.
>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message