crunch-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Ortiz <>
Subject Materializing map of avros
Date Fri, 06 May 2016 20:03:20 GMT

      In attempt to make my code a little bit easier to following, I am
attempting to materialize a PTable to a map and then pass it into another
DoFn.  Unfortunately, since the value is an Avro record, I am getting a
NotSerializableException out of the code when I try to use it.

     I attempting to get around this by converting the record into a
ByteBuffer with the avro utils, but lo and behold that's also not
Serializable.  Since I do not see a convenient way to wrap a byte array
with crunch, has anyone had any luck with any other approaches to getting a
crunch-compatible serialized avro object?

     David Ortiz

View raw message