spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Juan Martín Guillén <>
Subject Re: Issue in parallelization of CNN model using spark
Date Mon, 13 Jul 2020 14:09:13 GMT
 Hi Mukhtaj,
Parallelization on Spark is abstracted on the DataFrame.
You can run anything locally on the driver but to make it run in parallel on the cluster you'll
need to use the DataFrame abstraction.
You may want to check maxpumperla/elephas.

|  |  |


|  | 

Distributed Deep learning with Keras & Spark. Contribute to maxpumperla/elephas development
by creating an accou...



Regards,Juan Martín.

    El lunes, 13 de julio de 2020 08:59:35 ART, Mukhtaj Khan <> escribió:
 Dear Spark User
I am trying to parallelize the CNN (convolutional neural network) model using spark. I have
developed the model using python and Keras library. The model works fine on a single machine
but when we try on multiple machines, the execution time remains the same as sequential.Could
you please tell me that there is any built-in library for CNN to parallelize in spark framework.
Moreover, MLLIB does not have any support for CNN.Best regardsMukhtaj

View raw message