spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Luis Ángel Vicente Sánchez <>
Subject Re: proposal: replace lift-json with spray-json
Date Sun, 09 Feb 2014 19:50:08 GMT
spray-json future is not clear as spray is going to become akka-http and
the spray team is still deciding the future of spray-json... it may stay or
it may be combined with play-json to create a better library.

spray-json only relies on parboiled, a library mantained by the spray-team
itself. Right now it's performance is much worse than lift-json ( but that
would change when they finished parboiled2.

And alternative could be but you would bring scalaz as a
lift-json is a nice library, but Lift is a pretty heavyweight dependency to
track just for its JSON support.  (lift-json is relatively self-contained
as a dependency from an end-user's perspective, but downstream distributors
need to build all of Lift in order to package the JSON support.)  I
understand that this has come up before (cf. SPARK-883) and that the
uncertain future of JSON support in the Scala standard library is the
motivator for relying on an external library.

I'm proposing replacing lift-json in Spark with something more lightweight.
 I've evaluated apparent project liveness and dependency scope for most of
the current Scala JSON libraries and believe the best candidate is
spray-json (, the JSON library used by
the Spray HTTP toolkit. spray-json is Apache-licensed, actively developed,
and builds and works independently of Spray with only one external

It looks to me like a pretty straightforward change (although
JsonProtocol.scala would be a little more verbose since it couldn't use the
Lift JSON DSL), and I'd like to do it.  I'm writing now to ask for some
community feedback before making the change (and submitting a JIRA and PR).
 If no one has any serious objections (to the effort in general or to to
the choice of spark-json in particular), I'll go ahead and do it, but if
anyone has concerns, I'd be happy to discuss and address them before
getting started.


  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message