spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mayur Rustagi <mayur.rust...@gmail.com>
Subject Re: NLP with Spark
Date Wed, 12 Mar 2014 18:38:02 GMT
Would love to know if somebody has tried this, only possible problem I can
forsee is non-serializable libraries, else no reason it should not work.

Mayur Rustagi
Ph: +1 (760) 203 3257
http://www.sigmoidanalytics.com
@mayur_rustagi <https://twitter.com/mayur_rustagi>



On Wed, Mar 12, 2014 at 11:10 AM, shankark <shankark+sys@gmail.com> wrote:

> (apologies if this was sent out multiple times before)
>
> We are about to start a large-scale text-processing research project and
> are debating between two alternatives for our cluster -- Spark and Hadoop.
> I've researched possibilities of using NLTK with Hadoop and see that
> there's some precedent (
> http://blog.cloudera.com/blog/2010/03/natural-language-processing-with-hadoop-and-python/).
> I wanted to know how easy it might be to use NLTK with pyspark, or if
> scalanlp is mature enough to be used with the Scala API for Spark/mllib.
>
> Thanks!
>
> ------------------------------
> View this message in context: NLP with Spark<http://apache-spark-user-list.1001560.n3.nabble.com/NLP-with-Spark-tp2612.html>
> Sent from the Apache Spark User List mailing list archive<http://apache-spark-user-list.1001560.n3.nabble.com/>at
Nabble.com.
>

Mime
View raw message