spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adrian Tanase <>
Subject Re: how to broadcast huge lookup table?
Date Sun, 04 Oct 2015 16:34:46 GMT
have a look at .transformWith, you can specify another RDD.

Sent from my iPhone

On 02 Oct 2015, at 21:50, "<>"
<<>> wrote:

I tried broadcasting a key-value rdd, but then I cannot perform any rdd-actions inside a map/foreach
function of another rdd.

any tips? If going into scala collections I end up with huge memory bottlenecks.


View raw message