spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nick Pentreath <nick.pentre...@gmail.com>
Subject PySpark - Dill serialization
Date Thu, 05 Dec 2013 12:49:55 GMT
Hi devs

I came across Dill (
http://trac.mystic.cacr.caltech.edu/project/pathos/wiki/dill) for Python
serialization. Was wondering if it may be a replacement to the cloudpickle
stuff (and remove that piece of code that needs to be maintained within
PySpark)?

Josh have you looked into Dill? Any thoughts?

N

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message