spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suman bharadwaj <>
Subject Giraph Vs SPARK
Date Thu, 23 Jan 2014 21:10:16 GMT

I might be wrong, but need your help.

My understanding in Giraph is that, it doesn't write the intermediate data
to disk while sending messages to different machines. But in SPARK, I see
that intermediate map outputs gets written to disk. Why does SPARK write
intermediate data to disk ?

What happens at reducer side ? Does SPARK write the data again to disk ?
How does it differ from Hadoop MR ?

Can't SPARK communicate everything in memory ?

If my understanding is wrong. Please do correct me.

Suman Bharadwaj S

View raw message