spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SK <>
Subject json parsing with json4s
Date Wed, 11 Jun 2014 22:26:41 GMT
I have the following piece of code that parses a json file and extracts the
age and TypeID
val p = sc.textFile(log_file)
                   .map(line => { parse(line) })
                   .map(json => 
                      {  val v1 = json \ "person" \ "age"
                         val v2 = json \ "Action" \ "Content" \ "TypeID"
                         (v1, v2)
p.foreach(r => println(r))

The result is:


1) How can I extract the values (i.e. without the JInt) ? I tried returning
(v1.toInt, v2.toInt) from the map but got a compilation error stating that
toInt is not a valid operation.

2) I would also like to know how  I can filter the above tuples based on the
age values. For e.g. I added the following after the second map operation:

  p.filter(tup => tup._1 > 20)

I got a compilation errror: value > is not a member of org.json4s.JValue

Thanks for your help.


View this message in context:
Sent from the Apache Spark User List mailing list archive at

View raw message