spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dongjoon Hyun <dongjoon.h...@gmail.com>
Subject Re: [VOTE] SPIP: Add FunctionCatalog
Date Tue, 09 Mar 2021 05:44:54 GMT
+1 (binding)

Thank you, Ryan.

Bests,
Dongjoon.


On Mon, Mar 8, 2021 at 5:20 PM Chao Sun <sunchao@apache.org> wrote:

> +1 (non-binding)
>
> On Mon, Mar 8, 2021 at 5:13 PM John Zhuge <jzhuge@apache.org> wrote:
>
>> +1 (non-binding)
>>
>> On Mon, Mar 8, 2021 at 4:32 PM Holden Karau <holden@pigscanfly.ca> wrote:
>>
>>> +1 (binding)
>>>
>>> On Mon, Mar 8, 2021 at 3:56 PM Ryan Blue <blue@apache.org> wrote:
>>>
>>>> Hi everyone, I’d like to start a vote for the FunctionCatalog design
>>>> proposal (SPIP).
>>>>
>>>> The proposal is to add a FunctionCatalog interface that can be used to
>>>> load and list functions for Spark to call. There are interfaces for scalar
>>>> and aggregate functions.
>>>>
>>>> In the discussion we’ve come to consensus and I’ve updated the design
>>>> doc to match how functions will be called:
>>>>
>>>> In addition to produceResult(InternalRow), which is optional,
>>>> functions can define produceResult methods with arguments that are
>>>> Spark’s internal data types, like UTF8String. Spark will prefer these
>>>> methods when calling the UDF using codgen.
>>>>
>>>> I’ve also updated the AggregateFunction interface and merged it with
>>>> the partial aggregate interface because Spark doesn’t support non-partial
>>>> aggregates.
>>>>
>>>> The full SPIP doc is here:
>>>> https://docs.google.com/document/d/1PLBieHIlxZjmoUB0ERF-VozCRJ0xw2j3qKvUNWpWA2U/edit#heading=h.82w8qxfl2uwl
>>>>
>>>> Please vote on the SPIP in the next 72 hours. Once it is approved, I’ll
>>>> do a final update of the PR and we can merge the API.
>>>>
>>>> [ ] +1: Accept the proposal as an official SPIP
>>>> [ ] +0
>>>> [ ] -1: I don’t think this is a good idea because …
>>>> --
>>>> Ryan Blue
>>>>
>>> --
>>> Twitter: https://twitter.com/holdenkarau
>>> Books (Learning Spark, High Performance Spark, etc.):
>>> https://amzn.to/2MaRAG9  <https://amzn.to/2MaRAG9>
>>> YouTube Live Streams: https://www.youtube.com/user/holdenkarau
>>>
>>
>>
>> --
>> John Zhuge
>>
>

Mime
View raw message