spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From manasdebashiskar <poorinsp...@gmail.com>
Subject Re: mapwithstate Hangs with Error cleaning broadcast
Date Wed, 02 Nov 2016 21:59:45 GMT
Yes, 
 In my case, my StateSpec had a small partition size. I increased the
numPartitions and the problem went away. (Details of why the problem was
happening in the first place is elided.)

 TL;DR
 StateSpec takes a "numPartitions" which can be set to high enough number.



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/mapwithstate-Hangs-with-Error-cleaning-broadcast-tp26500p27994.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe e-mail: user-unsubscribe@spark.apache.org


Mime
View raw message