spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francis.Hu <>
Subject help me: Out of memory when spark streaming
Date Fri, 16 May 2014 08:41:22 GMT
hi, All


I encountered OOM when streaming.

I send data to spark streaming through Zeromq at a speed of 600 records per
second, but the spark streaming only handle 10 records per 5 seconds( set it
in streaming program)

my two workers have 4 cores CPU and 1G RAM.

These workers always occur Out Of Memory after moments.

I tried to adjust JVM GC arguments to speed up GC process.  Actually, it
made a little bit change of performance, but workers finally occur OOM.


Is there any way to resolve it?


it would be appreciated if anyone can help me to get it fixed !





View raw message