spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vr spark <vrspark...@gmail.com>
Subject dataframe row list question
Date Thu, 11 Aug 2016 14:54:54 GMT
I have data which is json in this format

 myList: array
 |    |    |-- elem: struct
 |    |    |    |-- nm: string (nullable = true)
 |    |    |    |-- vList: array (nullable = true)
 |    |    |    |    |-- element: string (containsNull = true)


 from my kafka stream, i created a dataframe using sqlContext.jsonRDD
 Then registred it as registerTempTable
 selected mylist from this table and i see this output. It is a list of rows

[Row(nm=u'Apt', vList=[u'image']), Row(nm=u'Agent', vList=[u'Mozilla/5.0
']), Row(nm=u'Ip', vList=[u'xx.yy.106.25'])]

 My requirement is to get only rows with nm='IP' and its corresponding value
I would need IP, xx.yy.106.25


Please suggest

Mime
View raw message