spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Barney <jamesbarne...@gmail.com>
Subject Re: Extract all the values from describe
Date Mon, 08 Feb 2016 15:32:13 GMT
Hi Arunkumar,
>From the scala documentation it's recommended to use the agg function for
performing any actual statistics programmatically on your data.
df.describe() is meant only for data exploration.

See Aggregator here:
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.Aggregator



On Mon, Feb 8, 2016 at 5:24 AM, Arunkumar Pillai <arunkumar1111@gmail.com>
wrote:

> hi
>
>
> I have  a dataframe df and  i use df.decribe() to get the stats value. but
> not able to parse and extract all the individual information. Please help
>
> --
> Thanks and Regards
>         Arun
>

Mime
View raw message