flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Theodore Vasiloudis (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1932) Add L-BFGS to the optimization framework
Date Mon, 14 Mar 2016 11:31:33 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15193130#comment-15193130

Theodore Vasiloudis commented on FLINK-1932:

[~spkavulya] The main reason we didn't create a PR for this was the fact that we I am using
a call to .collect() to get the updated weight vector at each iteration,
which can be very slow with the the way Flink works currently.

There might be a way to implement this without the collect, but I wasn't able to figure something
In any case, one thing I would like us to do is compare this implementation with the Spark
one in  terms of performance in a cluster.
If we are much slower than Spark, then I wouldn't recommend merging this algorithm, until
we get the performance right. 

> Add L-BFGS to the optimization framework
> ----------------------------------------
>                 Key: FLINK-1932
>                 URL: https://issues.apache.org/jira/browse/FLINK-1932
>             Project: Flink
>          Issue Type: New Feature
>          Components: Machine Learning Library
>            Reporter: Till Rohrmann
>            Assignee: Theodore Vasiloudis
>              Labels: ML
> A good candidate to add to the new optimization framework could be L-BFGS [1, 2].
> Resources:
> [1] http://papers.nips.cc/paper/5333-large-scale-l-bfgs-using-mapreduce.pdf
> [2] http://en.wikipedia.org/wiki/Limited-memory_BFGS

This message was sent by Atlassian JIRA

View raw message