spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SK <skrishna...@gmail.com>
Subject json parsing with json4s
Date Wed, 11 Jun 2014 22:26:41 GMT
I have the following piece of code that parses a json file and extracts the
age and TypeID
 
val p = sc.textFile(log_file)
                   .map(line => { parse(line) })
                   .map(json => 
                      {  val v1 = json \ "person" \ "age"
                         val v2 = json \ "Action" \ "Content" \ "TypeID"
                         (v1, v2)
                      }
                    )
                 
p.foreach(r => println(r))

The result is:

(JInt(12),JInt(5))
(JInt(32),JInt(6))
(JInt(40),JInt(7))

1) How can I extract the values (i.e. without the JInt) ? I tried returning
(v1.toInt, v2.toInt) from the map but got a compilation error stating that
toInt is not a valid operation.

2) I would also like to know how  I can filter the above tuples based on the
age values. For e.g. I added the following after the second map operation:

  p.filter(tup => tup._1 > 20)

I got a compilation errror: value > is not a member of org.json4s.JValue

Thanks for your help.

    




--
View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/json-parsing-with-json4s-tp7430.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

Mime
View raw message