spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Charles O. Bajomo" <charles.baj...@pretechconsulting.co.uk>
Subject [Spark] Accumulators or count()
Date Wed, 01 Mar 2017 12:26:42 GMT
Hello everyone, 

I wanted to know if there is any benefit to using an acculumator over just executing a count()
on the whole RDD. There seems to be a lot of issues with accumulator during a stage failure
and also seems to be an issue rebuilding them if the application restarts from a checkpoint.
Anyone have any suggestions no this? 

Thanks 

Mime
View raw message