spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Koert Kuipers <ko...@tresata.com>
Subject Re: Job initialization performance of Spark standalone mode vs YARN
Date Tue, 04 Mar 2014 01:23:11 GMT
to be more precise, the difference depends on de-serialization overhead
from kryo for your data structures.


On Mon, Mar 3, 2014 at 8:21 PM, Koert Kuipers <koert@tresata.com> wrote:

> yes, tachyon is in memory serialized, which is not as fast as cached in
> memory in spark (not serialized). the difference really depends on your job
> type.
>
>
>
> On Mon, Mar 3, 2014 at 7:10 PM, polkosity <polkosity@gmail.com> wrote:
>
>> Thats exciting!  Will be looking into that, thanks Andrew.
>>
>> Related topic, has anyone had any experience running Spark on Tachyon
>> in-memory filesystem, and could offer their views on using it?
>>
>>
>>
>> --
>> View this message in context:
>> http://apache-spark-user-list.1001560.n3.nabble.com/Job-initialization-performance-of-Spark-standalone-mode-vs-YARN-tp2016p2265.html
>> Sent from the Apache Spark User List mailing list archive at Nabble.com.
>>
>
>

Mime
View raw message