flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1992) Add convergence criterion to SGD optimizer
Date Thu, 21 May 2015 08:20:59 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14553862#comment-14553862
] 

ASF GitHub Bot commented on FLINK-1992:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/692#discussion_r30781414
  
    --- Diff: flink-staging/flink-ml/src/test/scala/org/apache/flink/ml/optimization/GradientDescentITSuite.scala
---
    @@ -205,6 +190,61 @@ class GradientDescentITSuite extends FlatSpec with Matchers with
FlinkTestBase {
     
       }
     
    -  // TODO: Need more corner cases
    +  it should "terminate early if the convergence criterion is reached" in {
    +    // TODO(tvas): We need a better way to check the convergence of the weights.
    +    // Ideally we want to have a Breeze-like system, where the optimizers carry a history
and that
    +    // can tell us whether we have converged and at which iteration
    +
    +    val env = ExecutionEnvironment.getExecutionEnvironment
    +
    +    env.setParallelism(2)
    +
    +    val sgdEarlyTerminate = GradientDescent()
    +      .setConvergenceThreshold(1e2)
    +      .setStepsize(1.0)
    +      .setIterations(800)
    +      .setLossFunction(SquaredLoss())
    +      .setRegularizationType(NoRegularization())
    +      .setRegularizationParameter(0.0)
    +
    +    val inputDS = env.fromCollection(data)
    +
    +    val weightDSEarlyTerminate = sgdEarlyTerminate.optimize(inputDS, None)
    +
    +    val weightListEarly: Seq[WeightVector] = weightDSEarlyTerminate.collect()
    +
    +    weightListEarly.size should equal(1)
    +
    +    val weightVectorEarly: WeightVector = weightListEarly.head
    +    val weightsEarly = weightVectorEarly.weights.asInstanceOf[DenseVector].data
    +    val weight0Early = weightVectorEarly.intercept
    +
    +    val sgdNoConvergence = GradientDescent()
    +      .setStepsize(1.0)
    --- End diff --
    
    If this solves the problem, then go for it.


> Add convergence criterion to SGD optimizer
> ------------------------------------------
>
>                 Key: FLINK-1992
>                 URL: https://issues.apache.org/jira/browse/FLINK-1992
>             Project: Flink
>          Issue Type: Improvement
>          Components: Machine Learning Library
>            Reporter: Till Rohrmann
>            Assignee: Theodore Vasiloudis
>            Priority: Minor
>              Labels: ML
>             Fix For: 0.9
>
>
> Currently, Flink's SGD optimizer runs for a fixed number of iterations. It would be good
to support a dynamic convergence criterion, too.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message