spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From sujeet jog <sujeet....@gmail.com>
Subject local Vs Standalonecluster production deployment
Date Sat, 28 May 2016 15:42:58 GMT
Hi,

I have a question w.r.t  production deployment mode of spark,

I have 3 applications which i would like to run independently on a single
machine, i need to run the drivers in the same machine.

The amount of resources i have is also limited, like 4- 5GB RAM , 3 - 4
cores.

For deployment in standalone mode : i believe i need

1 Driver JVM,  1 worker node ( 1 executor )
1 Driver JVM,  1 worker node ( 1 executor )
1 Driver JVM,  1 worker node ( 1 executor )

The issue here is i will require 6 JVM running in parallel, for which i do
not have sufficient CPU/MEM resources,


Hence i was looking more towards a local mode deployment mode, would like
to know if anybody is using local mode where Driver + Executor run in a
single JVM in production mode.

Are there any inherent issues upfront using local mode for production base
systems.?..

Mime
View raw message