spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Will Benton <>
Subject proposal: replace lift-json with spray-json
Date Sun, 09 Feb 2014 19:28:08 GMT
lift-json is a nice library, but Lift is a pretty heavyweight dependency to track just for
its JSON support.  (lift-json is relatively self-contained as a dependency from an end-user's
perspective, but downstream distributors need to build all of Lift in order to package the
JSON support.)  I understand that this has come up before (cf. SPARK-883) and that the uncertain
future of JSON support in the Scala standard library is the motivator for relying on an external

I'm proposing replacing lift-json in Spark with something more lightweight.  I've evaluated
apparent project liveness and dependency scope for most of the current Scala JSON libraries
and believe the best candidate is spray-json (, the JSON
library used by the Spray HTTP toolkit. spray-json is Apache-licensed, actively developed,
and builds and works independently of Spray with only one external dependency.

It looks to me like a pretty straightforward change (although JsonProtocol.scala would be
a little more verbose since it couldn't use the Lift JSON DSL), and I'd like to do it.  I'm
writing now to ask for some community feedback before making the change (and submitting a
JIRA and PR).  If no one has any serious objections (to the effort in general or to to the
choice of spark-json in particular), I'll go ahead and do it, but if anyone has concerns,
I'd be happy to discuss and address them before getting started.


View raw message