spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hao Wang <wh.s...@gmail.com>
Subject Re: Kyro deserialisation error
Date Thu, 17 Jul 2014 10:13:37 GMT
Hi, all

Yes, it's a name of Wikipedia article. I am running WikipediaPageRank
example of Spark Bagels.
I am wondering whether there is any relation to buffer size of Kyro.

The page rank can be successfully finished, sometimes not because this kind
of Kyro exception happens too many times, which beats the maxTaskFailures.

I find this *Kyro exception: unable to find class *in my successful case,
too.


Regards,
Wang Hao(王灏)

CloudTeam | School of Software Engineering
Shanghai Jiao Tong University
Address:800 Dongchuan Road, Minhang District, Shanghai, 200240
Email:wh.sjtu@gmail.com


On Thu, Jul 17, 2014 at 4:44 PM, Sean Owen <sowen@cloudera.com> wrote:

> Not sure if this helps, but it does seem to be part of a name in a
> Wikipedia article, and Wikipedia is the data set. So something is
> reading this class name from the data.
>
> http://en.wikipedia.org/wiki/Carl_Fridtjof_Rode
>
> On Thu, Jul 17, 2014 at 9:40 AM, Tathagata Das
> <tathagata.das1565@gmail.com> wrote:
> > Seems like there is some sort of stream corruption, causing Kryo read to
> > read a weird class name from the stream (the name "arl Fridtjof Rode" in
> the
> > exception cannot be a class!).
> > Not sure how to debug this.
> >
> > @Patrick: Any idea?
>

Mime
View raw message