flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4883) Prevent UDFs implementations through Scala singleton objects
Date Wed, 09 Nov 2016 14:32:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15651100#comment-15651100
] 

ASF GitHub Bot commented on FLINK-4883:
---------------------------------------

Github user StefanRRichter commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2729#discussion_r87202166
  
    --- Diff: flink-java/src/main/java/org/apache/flink/api/java/DataSet.java ---
    @@ -181,7 +181,17 @@ protected void fillInType(TypeInformation<T> typeInfo) {
     		return this.type;
     	}
     
    +	/**
    +     *  1. Check if the function is implemented by a scala object. Checks only if scala
object function forbidden
    +     *    is not disabled in the [[org.apache.flink.api.common.ExecutionConfig]]
    +     *
    +     *  2. Returns a "closure-cleaned" version of the given function. Cleans only if
closure cleaning
    +     *     is not disabled in the [[org.apache.flink.api.common.ExecutionConfig]]
    +     */
     	public <F> F clean(F f) {
    --- End diff --
    
    I suggest to move this check in a separate method like:
    
    ```public <F> F checkNotSingleton(F f)```
    
    One method should typically have one single duty, and the duty of a cleaning function
should not be checking for singleton objects. Both functions could then be chained however
into `clean(check(f))`.


> Prevent UDFs implementations through Scala singleton objects
> ------------------------------------------------------------
>
>                 Key: FLINK-4883
>                 URL: https://issues.apache.org/jira/browse/FLINK-4883
>             Project: Flink
>          Issue Type: Bug
>            Reporter: Stefan Richter
>            Assignee: Renkai Ge
>
> Currently, user can create and use UDFs in Scala like this:
> {code}
> object FlatMapper extends RichCoFlatMapFunction[Long, String, (Long, String)] {
> ...
> }
> {code}
> However, this leads to problems as the UDF is now a singleton that Flink could use across
several operator instances, which leads to job failures. We should detect and prevent the
usage of singleton UDFs.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message