spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akhil Das <ak...@sigmoidanalytics.com>
Subject Re: Does anyone have experience with using Hadoop InputFormats?
Date Thu, 25 Sep 2014 07:09:31 GMT
Hi

org.apache.hadoop.io.Text class is not Serializable. So the best thing you
can try is to get the Text.java file locally and serialize (implements
Serializable) it and use it. I have solved one of the similar problem this
way.

Thanks
Best Regards

On Wed, Sep 24, 2014 at 11:28 PM, Steve Lewis <lordjoe2000@gmail.com> wrote:

> Hmmm - I have only tested in local mode but I got an
> java.io.NotSerializableException: org.apache.hadoop.io.Text
>  at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1180)
>  at
> java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1528)
>  at
> java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1493)
> Here are two classes - one will work one will not
> the mgf file is what they read
>
> showPairRDD simply print  the text read
> guaranteeSparkMaster calls sparkConf.setMaster("local"); if there is no
> master defined
>
> Perhaps I need to convert Text somewhere else but I certainly don't see
> where
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: user-unsubscribe@spark.apache.org
> For additional commands, e-mail: user-help@spark.apache.org
>

Mime
View raw message