spark-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hyukjin Kwon <gurwls...@gmail.com>
Subject Re: Revisiting Python / pandas UDF (new proposal)
Date Mon, 06 Jan 2020 13:52:37 GMT
I happened to propose a somewhat big refactoring PR as a preparation for
this.
Basically, grouping all related codes into one sub-package since currently
all pandas and PyArrow related codes are here and there.
I would appreciate if you guys can review and give some feedback.

https://github.com/apache/spark/pull/27109

Thanks!


2020년 1월 4일 (토) 오전 5:11, Li Jin <ice.xelloss@gmail.com>님이 작성:

> Hyukjin,
>
> Thanks for putting this together. I took a look at the proposal and left
> some comments. At the high level I like using type hints to specify
> input/output types but not so use about type hints for cordiality. I have
> commented on more details in the doc.
>
> Li
>
> On Thu, Jan 2, 2020 at 9:42 AM Li Jin <ice.xelloss@gmail.com> wrote:
>
>> I am going to review this carefully today. Thanks for the work!
>>
>> Li
>>
>> On Wed, Jan 1, 2020 at 10:34 PM Hyukjin Kwon <gurwls223@gmail.com> wrote:
>>
>>> Thanks for comments Maciej - I am addressing them.
>>> adding Li Jin too.
>>>
>>> I plan to proceed this late this week or early next week to make it on
>>> time before code freeze.
>>> I am going to pretty actively respond so please give feedback if there's
>>> any :-).
>>>
>>>
>>>
>>> 2019년 12월 30일 (월) 오후 6:45, Hyukjin Kwon <gurwls223@gmail.com>님이
작성:
>>>
>>>> Hi all,
>>>>
>>>> I happen to come up with another idea about pandas redesign.
>>>> Thanks Reynold, Bryan, Xiangrui, Takuya and Tim for offline discussions
>>>> and
>>>> helping me to write this proposal.
>>>>
>>>> Please take a look and let me know what you guys think.
>>>>
>>>> -
>>>> https://docs.google.com/document/d/1-kV0FS_LF2zvaRh_GhkV32Uqksm_Sq8SvnBBmRyxm30/edit?usp=sharing
>>>> - https://issues.apache.org/jira/browse/SPARK-28264
>>>>
>>>> I know it's a holiday season but please have some time to take a look so
>>>> we can make it on time before code freeze (31st Jan).
>>>>
>>>>

Mime
View raw message