spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chip Senkbeil <>
Subject Re: SVD in pyspark ?
Date Sun, 25 Jan 2015 19:32:24 GMT
Hi Andreas,

With regard to the notebook interface,  you can use the Spark Kernel ( as the backend for an IPython 3.0
notebook. The kernel is designed to be the foundation for interactive
applications connecting to Apache Spark and uses the IPython 5.0 message
protocol - used by IPython 3.0 - to communicate.

See the getting started section here:

It discusses getting IPython connected to a Spark Kernel. If you have any
more questions, feel free to ask!

Chip Senkbeil
IBM Emerging Technologies Software Engineer

On Sun Jan 25 2015 at 1:12:32 PM Andreas Rhode <> wrote:

> Is the distributed SVD functionality exposed to Python yet?
> Seems it's only available to scala or java, unless I am missing something,
> looking for a pyspark equivalent to
> org.apache.spark.mllib.linalg.SingularValueDecomposition
> In case it's not there yet, is there a way to make a wrapper to call from
> python into the corresponding java/scala code? The reason for using python
> instead of just directly  scala is that I like to take advantage of the
> notebook interface for visualization.
> As a side, is there a inotebook like interface for the scala based REPL?
> Thanks
> Andreas
> --
> View this message in context: http://apache-spark-user-list.
> Sent from the Apache Spark User List mailing list archive at
> ---------------------------------------------------------------------
> To unsubscribe, e-mail:
> For additional commands, e-mail:

View raw message