spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aureliano Buendia <buendia...@gmail.com>
Subject Re: Error reading HDFS file using spark 0.9.0 / hadoop 2.2.0 - incompatible protobuf 2.5 and 2.4.1
Date Fri, 21 Mar 2014 22:49:56 GMT
On Tue, Mar 18, 2014 at 12:56 PM, Ognen Duzlevski <
ognen@plainvanillagames.com> wrote:

>
> On 3/18/14, 4:49 AM, dmpour23@gmail.com wrote:
>
>> On Sunday, 2 March 2014 19:19:49 UTC+2, Aureliano Buendia  wrote:
>>
>>> Is there a reason for spark using the older akka?
>>>
>>>
>>>
>>>
>>> On Sun, Mar 2, 2014 at 1:53 PM, 1esha <alexey.r...@gmail.com> wrote:
>>>
>>> The problem is in akka remote. It contains files compiled with 2.4.*.
>>> When
>>>
>>> you run it with 2.5.* in classpath it fails like above.
>>>
>>>
>>>
>>> Looks like moving to akka 2.3 will solve this issue. Check this issue -
>>>
>>> https://www.assembla.com/spaces/akka/tickets/3154-use-
>>> protobuf-version-2-5-0#/activity/ticket:
>>>
>>>
>>> Is the solution to exclude the  2.4.*. dependency on protobuf or will
>>> thi produce more complications?
>>>
>> I am not sure I remember what the context was around this but I run 0.9.0
> with hadoop 2.2.0 just fine.
>

The problem is that spark depends on an older version of akka, which
depends on an older version of protobuf (2.4).

This means people cannot use protobuf 2.5 with spark.


> Ognen
>

Mime
View raw message