spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "wyj"<...@meitu.com>
Subject Streaming performance would be better with input than without
Date Tue, 01 Nov 2016 09:32:48 GMT
<div style="font:14px/1.5 'Lucida Grande', '微软雅黑';color:#333;"><p style="font:14px/1.5
'Lucida Grande';margin:0;">Dear&nbsp;<span style="font-family: 'Lucida Grande',
微软雅黑;">Community,</span></p><p style="font:14px/1.5 'Lucida Grande';margin:0;"><span
style="font-family: 'Lucida Grande', 微软雅黑;"><br></span></p><p
style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 14px; line-height:
1.5; margin: 0px;">I am researching Spark Streaming for my team, and I am a freshman in
this field. Using kafka_wordcount.py(<a href="https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/kafka_wordcount.py">https://github.com/apache/spark/blob/master/examples/src/main/python/streaming/kafka_wordcount.py</a>)
and setting batchDuration=250ms or 500ms, I notice that there is a quite interesting scenario
that the processing time and total delay are lower when I put data to it than no input. I
am curious about why this would happen. The version of Spark is 1.6.1. &nbsp;Attached
are screenshot about those scenarios. &nbsp;<span style="font-family: arial; font-size:
13px; orphans: 2; widows: 2;">Your prompt reply would be highly appreciated</span><span
style="font-family: arial; font-size: 13px; orphans: 2; widows: 2;">.</span></p><p
style="font-style: normal; font-variant: normal; font-weight: normal; font-size: 14px; line-height:
1.5; margin: 0px;"><span style="font-family: arial; font-size: 13px; orphans: 2; widows:
2;"><br></span></p><p style="font-style: normal; font-variant: normal;
font-weight: normal; font-size: 14px; line-height: 1.5; margin: 0px;"><span style="font-family:
'Lucida Grande', 'Lucida Sans Unicode', sans-serif;">Sincerely</span></p><div><div
style="font-style: normal; font-variant: normal; font-size: 14px; line-height: 1.5; font-family:
'Lucida Grande';"><hr color="#b5c4df" size="1" align="left" style="font-weight: normal;
width: 210px; height: 1px; font-family: 'Lucida Grande', 'Lucida Sans Unicode', sans-serif
!important;"><div style="font-family: 'Lucida Grande', 'Lucida Sans Unicode', sans-serif
!important;"><div style="margin: 10px; font-size: 10pt;"><div style="margin: 10px;
font-size: 10pt;"><div style="font-size: 14px; line-height: 21px;"><font color="#136dba"><b>Frank</b></font></div><div
style="font-weight: normal; font-size: 14px; line-height: 21px;"><p class="MsoNormal"
style="margin: 0px 0cm;"><span lang="EN-US"><img src="http://pcl.dl.meitu.com/image001.png"
modifysize="100%" style="width: 233px; height: 80px;"></span></p><p class="MsoNormal"
style="margin: 0px 0cm;"></p><p class="MsoNormal" style="margin: 0px 0cm; line-height:
16pt; text-align: justify; font-size: 10.5pt;"><b><span lang="EN-US" style="font-size:
10pt;">Meitu, Inc.</span></b><span lang="EN-US"></span></p><p
class="MsoNormal" style="margin: 0px 0cm;"><b style="background-color: window;"><span
lang="EN-US" style="font-size: 10pt;">Mobile:</span></b>+86-<span lang="EN-US">18810578943</span></p><p
class="MsoNormal" style="margin: 0px 0cm;"><b style="background-color: window;"><span
lang="EN-US" style="font-size: 10pt;">Email</span></b>:<a href="mailto:wyj@meitu.com"
title="mailto:wyj@meitu.com">wyj@meitu.com</a></p><div style="line-height:
16pt;"><span style="line-height: 22px; font-size: 10.5pt;">Room 305, SOHU Network
Tower, Yard No.1, Zhongguancundong St., Haidian District, Beijing</span></div></div></div></div></div></div></div></div>
Mime
View raw message