spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joseph K. Bradley (JIRA)" <>
Subject [jira] [Commented] (SPARK-5564) Support sparse LDA solutions
Date Mon, 02 Mar 2015 17:59:04 GMT


Joseph K. Bradley commented on SPARK-5564:

It would be interesting to see comparisons between the two, but I don't have a good sense
of which would be more efficient.

{quote} I am assuming here that LDA architecture is a bipartite graph with nodes as docs/words
and there are counts on each edge {quote}
--> You're correct.

> Support sparse LDA solutions
> ----------------------------
>                 Key: SPARK-5564
>                 URL:
>             Project: Spark
>          Issue Type: Improvement
>          Components: MLlib
>    Affects Versions: 1.3.0
>            Reporter: Joseph K. Bradley
> Latent Dirichlet Allocation (LDA) currently requires that the priors’ concentration
parameters be > 1.0.  It should support values > 0.0, which should encourage sparser
topics (phi) and document-topic distributions (theta).
> For EM, this will require adding a projection to the M-step, as in: Vorontsov and Potapenko.
"Tutorial on Probabilistic Topic Modeling : Additive Regularization for Stochastic Matrix
Factorization." 2014.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message