spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mark Hamstra <m...@clearstorydata.com>
Subject Re: proposal: replace lift-json with spray-json
Date Sun, 09 Feb 2014 23:35:49 GMT
The JSON handling in the Rapture I/O <http://rapture.io/jsonSupport> library
is also pretty interesting, but I have no idea what its performance now is
or is likely to be, and code maturity is an issue with this project.


On Sun, Feb 9, 2014 at 3:06 PM, Pascal Voitot Dev <
pascal.voitot.dev@gmail.com> wrote:

> Hey andy ;)
>
> pickling would be cool for raw serialization/deserialization (not custom
> validations naturally).
> Anyway, as said in a previous mail on this mailing list, pickling can't be
> integrated now until we modify serializers to carry more type information
> so that pickling macros can be compiled...
>
> regards
> Pascal
>
> On Sun, Feb 9, 2014 at 9:09 PM, andy petrella <andy.petrella@gmail.com
> >wrote:
>
> > Pickling ?
> > At least, Heather did some benchmarks as well and the computability plus
> > its automation thanks to macro are two encouraging features.
> > Also, it is low level enough to allow some de/serialisation optimization.
> >
> > The ooyala blog doesn't include it already, but I know his author knows
> > about Pickling and I guess t best the time ran over :-)
> >
> > Andy
> >  Le 9 févr. 2014 20:50, "Luis Ángel Vicente Sánchez" <
> > langel.groups@gmail.com> a écrit :
> >
> > > spray-json future is not clear as spray is going to become akka-http
> and
> > > the spray team is still deciding the future of spray-json... it may
> stay
> > or
> > > it may be combined with play-json to create a better library.
> > >
> > > spray-json only relies on parboiled, a library mantained by the
> > spray-team
> > > itself. Right now it's performance is much worse than lift-json (
> > > http://engineering.ooyala.com/blog/comparing-scala-json-libraries) but
> > > that
> > > would change when they finished parboiled2.
> > >
> > > And alternative could be argonaut.io but you would bring scalaz as a
> > > dependency.
> > > lift-json is a nice library, but Lift is a pretty heavyweight
> dependency
> > to
> > > track just for its JSON support.  (lift-json is relatively
> self-contained
> > > as a dependency from an end-user's perspective, but downstream
> > distributors
> > > need to build all of Lift in order to package the JSON support.)  I
> > > understand that this has come up before (cf. SPARK-883) and that the
> > > uncertain future of JSON support in the Scala standard library is the
> > > motivator for relying on an external library.
> > >
> > > I'm proposing replacing lift-json in Spark with something more
> > lightweight.
> > >  I've evaluated apparent project liveness and dependency scope for most
> > of
> > > the current Scala JSON libraries and believe the best candidate is
> > > spray-json (https://github.com/spray/spray-json), the JSON library
> used
> > by
> > > the Spray HTTP toolkit. spray-json is Apache-licensed, actively
> > developed,
> > > and builds and works independently of Spray with only one external
> > > dependency.
> > >
> > > It looks to me like a pretty straightforward change (although
> > > JsonProtocol.scala would be a little more verbose since it couldn't use
> > the
> > > Lift JSON DSL), and I'd like to do it.  I'm writing now to ask for some
> > > community feedback before making the change (and submitting a JIRA and
> > PR).
> > >  If no one has any serious objections (to the effort in general or to
> to
> > > the choice of spark-json in particular), I'll go ahead and do it, but
> if
> > > anyone has concerns, I'd be happy to discuss and address them before
> > > getting started.
> > >
> > >
> > > thanks,
> > > wb
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message