spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From José Raúl Pérez Rodríguez <>
Subject cache OS memory and spark usage of it
Date Tue, 10 Apr 2018 17:27:26 GMT

When I issue a "free -m" command in a host, I see a lot of memory used 
for cache in OS, however Spark Streaming is not able to request that 
memory for its usage, and it fail the execution due to not been able to 
launch executors.

What I understand of the OS memory cache (the one in "free -m" command 
result) is that, in practice is a free memory, because programs can 
request that memory for usage when needed, and OS "gives" the requested 
amount to the program. Is that right? If not, what is the behavior of OS 
cache? And what can spark do to use this memory?

Thanks a lot,


To unsubscribe e-mail:

View raw message