[ https://issues.apache.org/jira/browse/SPARK-4101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14229916#comment-14229916
]
Ilya Ganelin commented on SPARK-4101:
-------------------------------------
Hu Peter - did you have an algorithm in mind for doing the word analogy? I saw a brief mention
here:
https://code.google.com/p/word2vec/
"
It was recently shown that the word vectors capture many linguistic regularities, for example
vector operations vector('Paris') - vector('France') + vector('Italy') results in a vector
that is very close to vector('Rome'), and vector('king') - vector('man') + vector('woman')
is close to vector('queen') [3, 1]. You can try out a simple demo by running demo-analogy.sh.
"
Is that what you had in mind or were you thinking of another approach?
> [MLLIB] Improve API in Word2Vec model
> -------------------------------------
>
> Key: SPARK-4101
> URL: https://issues.apache.org/jira/browse/SPARK-4101
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.1.0
> Reporter: Peter Rudenko
> Priority: Minor
>
> 1) Would be nice to be able to retrieve underlying model map, to be able to work with
it after (make an RDD, persist/load, online train, etc.). (Done by [SPARK-4582|https://issues.apache.org/jira/browse/SPARK-4582]
)
> 2) Be able to extend Word2VecModel to add custom functionality (like add analogyWords(w1:
String, w2: String, target: String, num: Int) method, which returns n words that relates to
target as w1 to w2).
> 3) Make cosineSimilarity method public to be able to reuse it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org
|