spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ilya Ganelin (JIRA)" <>
Subject [jira] [Commented] (SPARK-4101) [MLLIB] Improve API in Word2Vec model
Date Mon, 01 Dec 2014 15:48:13 GMT


Ilya Ganelin commented on SPARK-4101:

Hu Peter - did you have an algorithm in mind for doing the word analogy? I saw a brief mention

It was recently shown that the word vectors capture many linguistic regularities, for example
vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector
that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman')
is close to vector('queen') [3, 1]. You can try out a simple demo by running

Is that what you had in mind or were you thinking of another approach?

> [MLLIB] Improve API in Word2Vec model
> -------------------------------------
>                 Key: SPARK-4101
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 1.1.0
>            Reporter: Peter Rudenko
>            Priority: Minor
> 1) Would be nice to be able to retrieve underlying model map, to be able to work with
it after (make an RDD, persist/load,  online train, etc.). (Done by [SPARK-4582|]
> 2) Be able to extend Word2VecModel to add custom functionality (like add analogyWords(w1:
String, w2: String, target: String, num: Int) method, which returns n words that relates to
target as w1 to w2).
> 3) Make cosineSimilarity method public to be able to reuse it. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message