spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marcelo Vanzin <>
Subject Re: Kubernetes backend and docker images
Date Mon, 08 Jan 2018 21:44:22 GMT
On Mon, Jan 8, 2018 at 1:39 PM, Matt Cheah <> wrote:
> We would still want images to be able to be uniquely specified for the
> driver vs. the executors. For example, not all of the libraries required on
> the driver may be required on the executors, so the user would want to
> specify a different custom driver image from their custom executor image.

Are you saying that we should *require* different images for driver
and executor, as is the case today, or that we should *allow*
different images, but default to the same, as I'm proposing?

I see zero reason to require different images. While it's true that
the driver may need more libraries than the executor, 99% of the time
it's ok to just have those libraries everywhere - it makes
configuration easier and doesn't do any harm.


To unsubscribe e-mail:

View raw message