spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From nilesh <>
Subject Re: New API for TFIDF generation in Spark 1.1.0
Date Thu, 09 Oct 2014 23:16:00 GMT
Did some digging in the documentation. Looks like the IDFModel.transform only
accepts RDD as an input,
and not individual elements. Is this a bug? I am saying this because
HashingTF.transform accepts both RDD as well as vector elements as its

>From your post replying to Jatin, looks like you are using
IDFModel.transform with individual elements going in as inputs through the
map function. This is illegal as per the Spark documentation as well as from
what I am seeing in my prototyping.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message