spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ilya Ganelin (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-4101) [MLLIB] Improve API in Word2Vec model
Date Mon, 01 Dec 2014 15:48:13 GMT

    [ https://issues.apache.org/jira/browse/SPARK-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229916#comment-14229916
] 

Ilya Ganelin commented on SPARK-4101:
-------------------------------------

Hu Peter - did you have an algorithm in mind for doing the word analogy? I saw a brief mention
here: 
https://code.google.com/p/word2vec/

" 
It was recently shown that the word vectors capture many linguistic regularities, for example
vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector
that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman')
is close to vector('queen') [3, 1]. You can try out a simple demo by running demo-analogy.sh.
"

Is that what you had in mind or were you thinking of another approach?

> [MLLIB] Improve API in Word2Vec model
> -------------------------------------
>
>                 Key: SPARK-4101
>                 URL: https://issues.apache.org/jira/browse/SPARK-4101
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 1.1.0
>            Reporter: Peter Rudenko
>            Priority: Minor
>
> 1) Would be nice to be able to retrieve underlying model map, to be able to work with
it after (make an RDD, persist/load,  online train, etc.). (Done by [SPARK-4582|https://issues.apache.org/jira/browse/SPARK-4582]
)
> 2) Be able to extend Word2VecModel to add custom functionality (like add analogyWords(w1:
String, w2: String, target: String, num: Int) method, which returns n words that relates to
target as w1 to w2).
> 3) Make cosineSimilarity method public to be able to reuse it. 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message