spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Pentreath (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (SPARK-10041) Proposal of Parameter Server Interface for Spark
Date Fri, 24 Feb 2017 08:22:44 GMT

     [ https://issues.apache.org/jira/browse/SPARK-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nick Pentreath closed SPARK-10041.
----------------------------------
    Resolution: Won't Fix

> Proposal of Parameter Server Interface for Spark
> ------------------------------------------------
>
>                 Key: SPARK-10041
>                 URL: https://issues.apache.org/jira/browse/SPARK-10041
>             Project: Spark
>          Issue Type: New Feature
>          Components: ML, MLlib
>            Reporter: Yi Liu
>         Attachments: Proposal of Parameter Server Interface for Spark - v1.pdf
>
>
> Many large-scale machine learning algorithms (logistic regression, LDA, neural network,
etc.) have been built on top of Apache Spark. As discussed in SPARK-4590, a Parameter Server
(PS) architecture can greatly improve the scalability and efficiency for these large-scale
machine learning. There are some previous discussions on possible Parameter Server implementations
inside Spark (e.g., SPARK-6932). However, at this stage we believe it is more important for
the community to first define the proper interface of Parameter Server, which can be decoupled
from the actual PS implementations; consequently, it is possible to support different implementations
of Parameter Servers in Spark later. The attached document contains our initial proposal of
Parameter Server interface for ML algorithms on Spark, including data model, supported operations,
epoch support and possible Spark integrations.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message