spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ayan guha <guha.a...@gmail.com>
Subject Re: dataframe row list question
Date Sat, 13 Aug 2016 00:23:43 GMT
You can use dot notations.

select myList.vList from t where myList.nm=IP'

On Fri, Aug 12, 2016 at 9:11 AM, vr spark <vrspark123@gmail.com> wrote:

> Hi Experts,
>  Please suggest
>
> On Thu, Aug 11, 2016 at 7:54 AM, vr spark <vrspark123@gmail.com> wrote:
>
>>
>> I have data which is json in this format
>>
>>  myList: array
>>  |    |    |-- elem: struct
>>  |    |    |    |-- nm: string (nullable = true)
>>  |    |    |    |-- vList: array (nullable = true)
>>  |    |    |    |    |-- element: string (containsNull = true)
>>
>>
>>  from my kafka stream, i created a dataframe using sqlContext.jsonRDD
>>  Then registred it as registerTempTable
>>  selected mylist from this table and i see this output. It is a list of
>> rows
>>
>> [Row(nm=u'Apt', vList=[u'image']), Row(nm=u'Agent', vList=[u'Mozilla/5.0
>> ']), Row(nm=u'Ip', vList=[u'xx.yy.106.25'])]
>>
>>  My requirement is to get only rows with nm='IP' and its corresponding
>> value
>> I would need IP, xx.yy.106.25
>>
>>
>> Please suggest
>>
>
>


-- 
Best Regards,
Ayan Guha

Mime
View raw message