spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Georg Heiler <georg.kf.hei...@gmail.com>
Subject pip/conda distribution headless mode
Date Sun, 30 Aug 2020 22:06:05 GMT
Hi,

I want to use pyspark as distributed via conda in headless mode.
It looks like the hadoop binaries are bundles (= pip distributes a default
version)
https://stackoverflow.com/questions/63661404/bootstrap-spark-itself-on-yarn.

I want to ask if it would be possible to A) distribute the headless version
(=without hadoop) instead or B) distribute the headless version
additionally for pip & conda-forge distribution channels.

Best,
Georg

Mime
View raw message