spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gourav Sengupta <>
Subject Re: Deep Learning with Spark, what is your experience?
Date Sat, 04 May 2019 17:17:36 GMT
Try using MxNet and Horovod directly as well (I think that MXNet is worth a
try as well):

Ofcourse Tensorflow is backed by Google's advertisement team as well


On Sat, May 4, 2019 at 10:59 AM Riccardo Ferrari <> wrote:

> Hi list,
> I am trying to undestand if ti make sense to leverage on Spark as enabling
> platform for Deep Learning.
> My open question to you are:
>    - Do you use Apache Spark in you DL pipelines?
>    - How do you use Spark for DL? Is it just a stand-alone stage in the
>    workflow (ie data preparation script) or is it  more integrated
> I see a major advantage in leveraging on Spark as a unified entrypoint,
> for example you can easily abstract data sources and leverage on existing
> team skills for data pre-processing and training. On the flip side you may
> hit some limitations including supported versions and so on.
> What is your experience?
> Thanks!

View raw message