spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Feynman Liang <fli...@databricks.com>
Subject Re: required: org.apache.spark.streaming.dstream.DStream[org.apache.spark.mllib.linalg.Vector]
Date Mon, 29 Jun 2015 01:16:17 GMT
You are trying to predict on a DStream[LabeledPoint] (data + labels) but
predictOn expects a DStream[Vector] (just the data without the labels).

Try doing:

val unlabeledStream = labeledStream.map { x => x.features }
model.predictOn(unlabeledStream).print()

On Sun, Jun 28, 2015 at 6:03 PM, Arthur Chan <arthur.hk.chan@gmail.com>
wrote:

> also my Spark is 1.4
>
> On Mon, Jun 29, 2015 at 9:02 AM, Arthur Chan <arthur.hk.chan@gmail.com>
> wrote:
>
>>
>>
>> Hi,
>>
>>
>> line 99:    model.trainOn(labeledStream)
>>
>> line 100:     model.predictOn(labeledStream).print()
>>
>> line 101:    ssc.start()
>>
>> line 102:     ssc.awaitTermination()
>>
>>
>> Regards
>>
>> On Sun, Jun 28, 2015 at 10:53 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>
>>> Can you show us your code around line 100 ?
>>>
>>> Which Spark release are you compiling against ?
>>>
>>> Cheers
>>>
>>> On Sun, Jun 28, 2015 at 5:49 AM, Arthur Chan <arthur.hk.chan@gmail.com>
>>> wrote:
>>>
>>>> Hi,
>>>>
>>>> I am trying Spark with some sample programs,
>>>>
>>>>
>>>> In my code, the following items are imported:
>>>>
>>>> import
>>>> org.apache.spark.mllib.regression.{StreamingLinearRegressionWithSGD,
>>>> LabeledPoint}
>>>>
>>>> import
>>>> org.apache.spark.mllib.regression.{StreamingLinearRegressionWithSGD}
>>>>
>>>> import org.apache.spark.streaming.{Seconds, StreamingContext}
>>>>
>>>> import scala.util.Random
>>>>
>>>> I got following error:
>>>>
>>>> [error] StreamingModel.scala:100: type mismatch;
>>>>
>>>> [error]  found   :
>>>> org.apache.spark.streaming.dstream.DStream[org.apache.spark.mllib.regression.LabeledPoint]
>>>>
>>>> [error]  required:
>>>> org.apache.spark.streaming.dstream.DStream[org.apache.spark.mllib.linalg.Vector]
>>>>
>>>> [error]     model.predictOn(labeledStream).print()
>>>>
>>>> [error]                     ^
>>>>
>>>> [error] one error found
>>>>
>>>> [error] (compile:compile) Compilation failed
>>>>
>>>>
>>>> Any idea?
>>>>
>>>>
>>>> Regards
>>>>
>>>
>>>
>>
>

Mime
View raw message