spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Erik Erlandson <>
Subject Toward an "API" for spark images used by the Kubernetes back-end
Date Thu, 22 Mar 2018 00:04:29 GMT
During the review of the recent PR to remove use of the init_container from
kube pods as created by the Kubernetes back-end, the topic of documenting
the "API" for these container images also came up. What information does
the back-end provide to these containers? In what form? What assumptions
does the back-end make about the structure of these containers?  This
information is important in a scenario where a user wants to create custom
images, particularly if these are not based on the reference dockerfiles.

A related topic is deciding what such an API should look like.  For
example, early incarnations were based more purely on environment
variables, which could have advantages in terms of an API that is easy to
describe in a document.  If we document the current API, should we annotate
it as Experimental?  If not, does that effectively freeze the API?

We are interested in community input about possible customization use cases
and opinions on possible API designs!

View raw message