spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From V0lleyBallJunki3 <>
Subject Unable to broadcast a very large variable
Date Wed, 10 Apr 2019 09:06:31 GMT
   I have a 110 node cluster with each executor having 50 GB memory and I
want to broadcast a variable of 70GB with each machine have 244 GB of
memory. I am having difficulty doing that. I was wondering at what size is
it unwise to broadcast a variable. Is there a general rule of thumb?

Sent from:

To unsubscribe e-mail:

View raw message