spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MLnick <...@git.apache.org>
Subject [GitHub] incubator-spark pull request: [PySpark] Adding support for Sequenc...
Date Sat, 08 Feb 2014 15:36:30 GMT
Github user MLnick commented on the pull request:

    https://github.com/apache/incubator-spark/pull/263#issuecomment-34546862
  
    @laserson sorry this has gone quiet as I have been totally slammed with work stuff! 
    
    From my side I just need to clean up some of @JoshRosen comments. The main thing is the
default `toString` which will require bringing back the `PairMUTF8Deserializer` since `msgpack`
won't be used in that case (or perhaps just having `UTF8Deserializer` handle (string, string)
also.
    
    The other major missing piece is then the key and value wrapper approach, which would
be useful to get your thoughts on.
    
    Perhaps you could test the PR out on Avro/Parquet data, and see if it works, or if not
where it falls down that we may need to improve?


Mime
View raw message