flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hae Joon Lee (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-1731) Add kMeans clustering algorithm to machine learning library
Date Sun, 17 May 2015 12:11:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14547149#comment-14547149
] 

Hae Joon Lee edited comment on FLINK-1731 at 5/17/15 12:10 PM:
---------------------------------------------------------------

Hi, I am testing K-mean right now.
I faced an error "could not find implicit value for parameter"
I solved a lot of things except for this one.

* Error:(142, 53) could not find implicit value for parameter op: breeze.linalg.operators.OpDiv.Impl2[breeze.linalg.Vector[Double],Long,That].map(x
=> LabeledVector(x._1, x._2.asBreeze / 1L)).withForwardedFields("_1->id")

{code:title=KMeans.scala|borderStyle=solid}
val finalCentroids = centroids.iterate(numIterations) { currentCentroids =>
val newCentroids: DataSet[LabeledVector] = input
  .map(new SelectNearestCenterMapper).withBroadcastSet(currentCentroids, CENTROIDS)
  .map(x => (x.label, x.vector, 1L)).withForwardedFields("_1; _2")
  .groupBy(x => x._1)
  .reduce((p1, p2) => (p1._1, (p1._2.asBreeze + p2._2.asBreeze).fromBreeze, p1._3 + p2._3)).withForwardedFields("_1")
  .map(x => LabeledVector(x._1, x._2.asBreeze :/ x._3)).withForwardedFields("_1->id")
  newCentroids
}
{code}

As far as I know, the error "could not find implicit value for parameter" can be solved by
putting exact import class.
                                 
I think the error is from :/ operator, so I put 'import breeze.linalg.operators._' on import
line as well. but it does not work. 
Have you ever seen this kind of error before?




was (Author: philjjoon):
Hi, I am testing K-mean right now.
I faced an error "could not find implicit value for parameter"
I solved a lot of things except for this one.

* Error:(142, 53) could not find implicit value for parameter op: breeze.linalg.operators.OpDiv.Impl2[breeze.linalg.Vector[Double],Long,That].map(x
=> LabeledVector(x._1, x._2.asBreeze / 1L)).withForwardedFields("_1->id")

{code:title=KMeans.scala|borderStyle=solid}
val finalCentroids = centroids.iterate(numIterations) { currentCentroids =>
val newCentroids: DataSet[LabeledVector] = input
  .map(new SelectNearestCenterMapper).withBroadcastSet(currentCentroids, CENTROIDS)
  .map(x => (x.label, x.vector, 1L)).withForwardedFields("_1; _2")
  .groupBy(x => x._1)
  .reduce((p1, p2) => (p1._1, (p1._2.asBreeze + p2._2.asBreeze).fromBreeze, p1._3 + p2._3)).withForwardedFields("_1")
  .map(x => LabeledVector(x._1, x._2.asBreeze :/ x._3)).withForwardedFields("_1->id")
  newCentroids
}
{code}

As far as I know, the error "could not find implicit value for parameter" can be solved by
putting exact import class.
                                                    
I put 'import breeze.linalg.operators._' on import line as well. 
but it does not work.
Have you ever seen this kind of error before?



> Add kMeans clustering algorithm to machine learning library
> -----------------------------------------------------------
>
>                 Key: FLINK-1731
>                 URL: https://issues.apache.org/jira/browse/FLINK-1731
>             Project: Flink
>          Issue Type: New Feature
>          Components: Machine Learning Library
>            Reporter: Till Rohrmann
>            Assignee: Peter Schrott
>              Labels: ML
>
> The Flink repository already contains a kMeans implementation but it is not yet ported
to the machine learning library. I assume that only the used data types have to be adapted
and then it can be more or less directly moved to flink-ml.
> The kMeans++ [1] and the kMeans|| [2] algorithm constitute a better implementation because
the improve the initial seeding phase to achieve near optimal clustering. It might be worthwhile
to implement kMeans||.
> Resources:
> [1] http://ilpubs.stanford.edu:8090/778/1/2006-13.pdf
> [2] http://theory.stanford.edu/~sergei/papers/vldb12-kmpar.pdf



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message