spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chip Senkbeil <chip.senkb...@gmail.com>
Subject Re: SVD in pyspark ?
Date Sun, 25 Jan 2015 19:32:24 GMT
Hi Andreas,

With regard to the notebook interface,  you can use the Spark Kernel (
https://github.com/ibm-et/spark-kernel) as the backend for an IPython 3.0
notebook. The kernel is designed to be the foundation for interactive
applications connecting to Apache Spark and uses the IPython 5.0 message
protocol - used by IPython 3.0 - to communicate.

See the getting started section here:
https://github.com/ibm-et/spark-kernel/wiki/Getting-Started-with-the-Spark-Kernel

It discusses getting IPython connected to a Spark Kernel. If you have any
more questions, feel free to ask!

Signed,
Chip Senkbeil
IBM Emerging Technologies Software Engineer

On Sun Jan 25 2015 at 1:12:32 PM Andreas Rhode <m.a.rhode@gmail.com> wrote:

> Is the distributed SVD functionality exposed to Python yet?
>
> Seems it's only available to scala or java, unless I am missing something,
> looking for a pyspark equivalent to
> org.apache.spark.mllib.linalg.SingularValueDecomposition
>
> In case it's not there yet, is there a way to make a wrapper to call from
> python into the corresponding java/scala code? The reason for using python
> instead of just directly  scala is that I like to take advantage of the
> notebook interface for visualization.
>
> As a side, is there a inotebook like interface for the scala based REPL?
>
> Thanks
>
> Andreas
>
>
>
> --
> View this message in context: http://apache-spark-user-list.
> 1001560.n3.nabble.com/SVD-in-pyspark-tp21356.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>
>

Mime
View raw message