spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Pentreath (JIRA)" <>
Subject [jira] [Commented] (SPARK-10041) Proposal of Parameter Server Interface for Spark
Date Fri, 24 Feb 2017 08:21:44 GMT


Nick Pentreath commented on SPARK-10041:

I think it is safe to say this is not going to be part of MLlib core. I'll close this.

> Proposal of Parameter Server Interface for Spark
> ------------------------------------------------
>                 Key: SPARK-10041
>                 URL:
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML, MLlib
>            Reporter: Yi Liu
>         Attachments: Proposal of Parameter Server Interface for Spark - v1.pdf
> Many large-scale machine learning algorithms (logistic regression, LDA, neural network,
etc.) have been built on top of Apache Spark. As discussed in SPARK-4590, a Parameter Server
(PS) architecture can greatly improve the scalability and efficiency for these large-scale
machine learning. There are some previous discussions on possible Parameter Server implementations
inside Spark (e.g., SPARK-6932). However, at this stage we believe it is more important for
the community to first define the proper interface of Parameter Server, which can be decoupled
from the actual PS implementations; consequently, it is possible to support different implementations
of Parameter Servers in Spark later. The attached document contains our initial proposal of
Parameter Server interface for ML algorithms on Spark, including data model, supported operations,
epoch support and possible Spark integrations.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message