spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Platter <>
Subject R: Broadcast variables: when should I use them?
Date Mon, 26 Jan 2015 13:43:22 GMT

Yes, if they are not big, it's a good practice to broadcast them to avoid serializing them
each time you use clojure.


Inviata dal mio Windows Phone
Da: frodo777<>
Inviato: ‎26/‎01/‎2015 14:34
Oggetto: Broadcast variables: when should I use them?


I have a number of "static" Arrays and Maps in my Spark Streaming driver
They are simple collections, initialized with integer values and strings
directly in the code. There is no RDD/DStream involvement here.....
I do not expect them to contain more than 100 entries, each.
They are used in several subsequent parallel operations.

The question is:
Should I convert them into broadcast variables?

Thanks and regards.

View this message in context:
Sent from the Apache Spark User List mailing list archive at

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message