spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ruslan Dautkhanov <dautkha...@gmail.com>
Subject Re: TCP/IP speedup
Date Sun, 02 Aug 2015 01:26:59 GMT
If your network is bandwidth-bound, you'll see setting jumbo frames (MTU
9000)
may increase bandwidth up to ~20%.

http://docs.hortonworks.com/HDP2Alpha/index.htm#Hardware_Recommendations_for_Hadoop.htm
"Enabling Jumbo Frames across the cluster improves bandwidth"

If Spark workload is not network bandwidth-bound, I can see it'll be a few
percent to no improvement.



-- 
Ruslan Dautkhanov

On Sat, Aug 1, 2015 at 6:08 PM, Simon Edelhaus <edelhas@gmail.com> wrote:

> Hmmmm....
>
> 2% huh.
>
>
> -- ttfn
> Simon Edelhaus
> California 2015
>
> On Sat, Aug 1, 2015 at 3:45 PM, Mark Hamstra <mark@clearstorydata.com>
> wrote:
>
>> https://spark-summit.org/2015/events/making-sense-of-spark-performance/
>>
>> On Sat, Aug 1, 2015 at 3:24 PM, Simon Edelhaus <edelhas@gmail.com> wrote:
>>
>>> Hi All!
>>>
>>> How important would be a significant performance improvement to TCP/IP
>>> itself, in terms of
>>> overall job performance improvement. Which part would be most
>>> significantly accelerated?
>>> Would it be HDFS?
>>>
>>> -- ttfn
>>> Simon Edelhaus
>>> California 2015
>>>
>>
>>
>

Mime
View raw message