beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Baptiste Onofré (JIRA) <j...@apache.org>
Subject [jira] [Commented] (BEAM-621) Add MapValues and MapKeys functions
Date Thu, 29 Sep 2016 15:29:20 GMT

    [ https://issues.apache.org/jira/browse/BEAM-621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15533088#comment-15533088
] 

Jean-Baptiste Onofré commented on BEAM-621:
-------------------------------------------

About the same idea, it would be great to have {{MapElements}} working with POJO. For instance
if the {{PCollection}} contains POJO like:

{code}
class MyPojo implements Serializable {
  ...
  public String getName() { ... }
  public Integer getAge() { ... }
}
{code}

today, to map this POJO to just name, we have to use {{MapElements}} with a function. It could
be interesting to provide a convenient function like {{MapElements.with("name"))}} that will
use reflection to map on {{getName()}}.

> Add MapValues and MapKeys functions
> -----------------------------------
>
>                 Key: BEAM-621
>                 URL: https://issues.apache.org/jira/browse/BEAM-621
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model
>            Reporter: Jean-Baptiste Onofré
>            Assignee: Jean-Baptiste Onofré
>
> Currently, we have the {{MapElements}} {{PTransform}} that "convert" a {{PCollection}}
of {{KV}} to another {{PCollection}} (for instance {{String}}).
> A very classic mapping function is to just have the keys or values of {{KV}}.
> To do it currently, we can use {{MapElements}} or a generic {{ParDo}} (with {{DoFn}}).
> It would be helpful and reduce the user code to have {{MapValues}} and {{MapKeys}}. It
would take a {{PCollection}} of {{KV}}: {{MapKeys}} will map the input {{PCollection}} to
a {{PCollection}} of {{K}} and {{MapValues}} to a {{PCollection}} of {{V}}.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message