Is there an example somewhere on using LDA’s logPerplexity()/logLikelihood() functions to evaluate topic counts? The existing MLLib LDA examples show calling them, but I can’t find any documentation about how to interpret the outputs. Graphing the outputs for logs of perplexity and likelihood aren’t consistent with what I expected (perplexity increases and likelihood decreases as topics increase, which seem odd to me).
Thanks very much in advance! If I can figure this out, I can post example code online, so others can see how this process is done.