spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pedro <ski.rodrig...@gmail.com>
Subject Re: Variables outside of mapPartitions scope
Date Fri, 09 May 2014 20:13:15 GMT
Right now I am not using any class variables (references to this). All my
variables are created within the scope of the method I am running.

I did more debugging and found this strange behavior.
variables here
for loop
    mapPartitions call
        use variables here
    end mapPartitions
endfor

This will result in a serializable bug, but this won't

variables here
for loop
    create new references to variables here
    mapPartitions call
        use new reference variables here
    end mapPartitions
endfor



--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Variables-outside-of-mapPartitions-scope-tp5517p5528.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message