spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Georg Heiler <georg.kf.hei...@gmail.com>
Subject Re: pip/conda distribution headless mode
Date Mon, 31 Aug 2020 04:47:38 GMT
Many thanks.

Best,
Georg

Am Mo., 31. Aug. 2020 um 01:12 Uhr schrieb Xiao Li <lixiao@databricks.com>:

> Hi, Georg,
>
> This is being tracked by https://issues.apache.org/jira/browse/SPARK-32017 You
> can leave comments in the JIRA.
>
> Thanks,
>
> Xiao
>
> On Sun, Aug 30, 2020 at 3:06 PM Georg Heiler <georg.kf.heiler@gmail.com>
> wrote:
>
>> Hi,
>>
>> I want to use pyspark as distributed via conda in headless mode.
>> It looks like the hadoop binaries are bundles (= pip distributes a
>> default version)
>> https://stackoverflow.com/questions/63661404/bootstrap-spark-itself-on-yarn
>> .
>>
>> I want to ask if it would be possible to A) distribute the headless
>> version (=without hadoop) instead or B) distribute the headless version
>> additionally for pip & conda-forge distribution channels.
>>
>> Best,
>> Georg
>>
>
>
> --
> <https://databricks.com/sparkaisummit/north-america>
>

Mime
View raw message