spark-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Reynold Xin <r...@databricks.com>
Subject Re: Re: Multiple streaming aggregations in structured streaming
Date Mon, 21 Nov 2016 07:56:21 GMT
Can you use the approximate count distinct?


On Sun, Nov 20, 2016 at 11:51 PM, Xinyu Zhang <wszxyh@163.com> wrote:

>
> MapWithState is also very useful.
> I want to calculate UV in real time, but "distinct count" and "multiple
> streaming aggregations" are not supported.
> Is there any method to calculate real-time UV in the current version?
>
>
>
> At 2016-11-19 06:01:45, "Michael Armbrust" <michael@databricks.com> wrote:
>
> Doing this generally is pretty hard.  We will likely support algebraic
> aggregate eventually, but this is not currently slotted for 2.2.  Instead I
> think we will add something like mapWithState that lets users compute
> arbitrary stateful things.  What is your use case?
>
>
> On Wed, Nov 16, 2016 at 6:58 PM, wszxyh <wszxyh@163.com> wrote:
>
>> Hi
>>
>> Multiple streaming aggregations are not yet supported. When will it be
>> supported? Is it in the plan?
>>
>> Thanks
>>
>>
>>
>>
>
>
>
>
>

Mime
View raw message