crunch-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabriel Reid (Jira)" <j...@apache.org>
Subject [jira] [Updated] (CRUNCH-651) (WIP) Kotlin API for Crunch
Date Wed, 09 Oct 2019 18:29:00 GMT

     [ https://issues.apache.org/jira/browse/CRUNCH-651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gabriel Reid updated CRUNCH-651:
--------------------------------
    Fix Version/s:     (was: 1.0.0)

> (WIP) Kotlin API for Crunch
> ---------------------------
>
>                 Key: CRUNCH-651
>                 URL: https://issues.apache.org/jira/browse/CRUNCH-651
>             Project: Crunch
>          Issue Type: Improvement
>            Reporter: David Whiting
>            Assignee: David Whiting
>            Priority: Minor
>         Attachments: 0001-WIP-Kotlin-API-for-Crunch.patch
>
>
> Motivation: Kotlin has recently gained a lot of popularity and people are really starting
to talk about it and see its merits; and language-wise it should be perfect for writing data-transformation
code as it has better support for functional programming and useful type stuff than Java,
without the performance overhead and implicit mysteries of Scala.
> Attached is a patch for a proof-of-concept Kotlin API for Crunch. It makes heavy use
of inlined functions so that the compiled bytecode ends up looking a lot like standard Crunch
code (I ran it through a decompiler), and should be just as performant.
> It also uses reified type parameters to construct PTypes automatically from TypeReferences,
so in most cases it's not necessary to provide a PType for transformations.
> See the example job and unit tests to get a quick idea of what kind of thing is possible.
> I've decided to share an unfinished patch with you all so that I can get some feedback
if we think this is a useful addition to the Crunch project at all, and whether the general
approach seems sensible.
> Still left TODO (if we decide to finish this and add it):
> * PGroupedTable API
> * Extensible PType inference (I'm thinking something like register<MyType>(pTypeForMyType))
> * Support for data classes via kotlin-reflect
> * Fill in any other gaps (mapWithContext, counters, ... ?)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message