spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rapelly kartheek <>
Subject Resource allocations
Date Wed, 16 Jul 2014 15:52:01 GMT

I am trying to understand how the resource allocation happens in spark. I
understand the resourceOffer method in taskScheduler. This method takes
care of locality factor while allocating the resources. This resourceOffer
method gets invoked by the corresponding cluster manager.

I am working on stand-alone spark cluster. But I am not able to locate
starting point of the resource allocation. I want to understand from end to
end how exactly resource allocations happens, given a new Application to
the Spark. Also, apart from mesos, there is a folder called Local in the
Scheduler. Can someone tell me which all files should I look into to
understand the resource allocation in stand-alone spark cluster and what is
this "Local" for?

Thanks in advance!!

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message