spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From 陶 加涛 <>
Subject Re: About introduce function sum0 to Spark
Date Tue, 23 Oct 2018 06:58:04 GMT
The name is from Apache Calcite, And it doesn’t matter, we can introduce our own.

Aron Tao

发件人: Mark Hamstra <>
日期: 2018年10月23日 星期二 12:28
收件人: "" <>
抄送: dev <>
主题: Re: About introduce function sum0 to Spark

That's a horrible name. This is just a fold.

On Mon, Oct 22, 2018 at 7:39 PM 陶 加涛 <<>>
Hi, in calcite, has the concept of sum0, here I quote the definition of sum0:

Sum0 is an aggregator which returns the sum of the values which
go into it like Sum. It differs in that when no non null values
are applied zero is returned instead of null..

One scenario is that we can use sum0 to implement pre-calculation count(pre-calculation system
like Apache Kylin).

It is very easy in Spark to implement sum0, if community consider this is necessary, I would
like to open a JIRA and implement this.

Aron Tao

View raw message