spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wkhapy <450533...@qq.com>
Subject spark sql get result time larger than compute Duration
Date Mon, 12 Mar 2018 03:14:26 GMT
<http://apache-spark-developers-list.1001551.n3.nabble.com/file/t2948/Kl0VG.png> 
get result 1.67s 


compute cost 0.2s 
https://i.stack.imgur.com/wpLV3.png

below is sql select event_date, dim ,concat_ws('|',collect_list(result))
result from ( select event_day event_date , '' dim
,concat_ws(',',result,event) result from ( select event_day
,event,count(uid) result from (select uid,event_day ,event ,uid from
usereventattr1 where ( city ='a' ) and ( event='WENJUANWANG__SUBMIT' ) )
usereventattrchild group by event ,event_day union all select event_day
,event,count(uid) result from (select uid,event_day ,event ,uid from
usereventattr1 where ( city ='a' ) and ( event='WECHAT__SUBSCRIBE' ) )
usereventattrchild group by event ,event_day ) xx) ab group by
dim,event_date

sql explain 
<http://apache-spark-developers-list.1001551.n3.nabble.com/file/t2948/6p0Em.png> 
explain get result also cost 1.4s
anybody kown why get result time large than compute 1.4s



--
Sent from: http://apache-spark-developers-list.1001551.n3.nabble.com/

---------------------------------------------------------------------
To unsubscribe e-mail: dev-unsubscribe@spark.apache.org


Mime
View raw message