flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3519) Subclasses of Tuples don't work if the declared type of a DataSet is not the descendant
Date Fri, 01 Apr 2016 18:48:25 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15222156#comment-15222156

ASF GitHub Bot commented on FLINK-3519:

Github user StephanEwen commented on the pull request:

    Thanks, I like the docs now!
    Sorry for nitpicking, but could you
      - Make the bullet list in the docs use the html tags (`<ul><li></li></ul>`)?
It does not render properly in the docs otherwise.
      - Write references to for example `DataSet<Tuple10>` as `DataStream<Tuple10>`/`DataSet<Tuple10>`?
I think the DataStream API is used more by now, so would be nice to have it in there.

> Subclasses of Tuples don't work if the declared type of a DataSet is not the descendant
> ---------------------------------------------------------------------------------------
>                 Key: FLINK-3519
>                 URL: https://issues.apache.org/jira/browse/FLINK-3519
>             Project: Flink
>          Issue Type: Bug
>          Components: Type Serialization System
>    Affects Versions: 1.0.0
>            Reporter: Gabor Gevay
>            Assignee: Gabor Gevay
>            Priority: Minor
> If I have a subclass of TupleN, then objects of this type will turn into TupleNs when
I try to use them in a DataSet<TupleN>.
> For example, if I have a class like this:
> {code}
> public static class Foo extends Tuple1<Integer> {
> 	public short a;
> 	public Foo() {}
> 	public Foo(int f0, int a) {
> 		this.f0 = f0;
> 		this.a = (short)a;
> 	}
> 	@Override
> 	public String toString() {
> 		return "(" + f0 + ", " + a + ")";
> 	}
> }
> {code}
> And then I do this:
> {code}
> env.fromElements(0,0,0).map(new MapFunction<Integer, Tuple1<Integer>>() {
> 	@Override
> 	public Tuple1<Integer> map(Integer value) throws Exception {
> 		return new Foo(5, 6);
> 	}
> }).print();
> {code}
> Then I don't have Foos in the output, but only Tuples:
> {code}
> (5)
> (5)
> (5)
> {code}
> The problem is caused by the TupleSerializer not caring about subclasses at all. I guess
the reason for this is performance: we don't want to deal with writing and reading subclass
tags when we have Tuples.
> I see three options for solving this:
> 1. Add subclass tags to the TupleSerializer: This is not really an option, because we
don't want to loose performance.
> 2. Document this behavior in the javadoc of the Tuple classes.
> 3. Make the Tuple types final: this would be the clean solution, but it is API breaking,
and the first victim would be Gelly: the Vertex and Edge types extend from tuples. (Note that
the issue doesn't appear there, because the DataSets there always have the type of the descendant
> When deciding between 2. and 3., an important point to note is that if you have your
class extend from a Tuple type instead of just adding the f0, f1, ... fields manually in the
hopes of getting the performance boost associated with Tuples, then you are out of luck: the
PojoSerializer will kick in anyway when the declared types of your DataSets are the descendant
> If someone knows about a good reason to extend from a Tuple class, then please comment.
> For 2., this is a suggested wording for the javadoc of the Tuple classes:
> Warning: Please don't subclass Tuple classes, but if you do, then be sure to always declare
the element type of your DataSets to your descendant type. (That is, if you have a "class
A extends Tuple2", then don't use instances of A in a DataSet<Tuple2>, but use DataSet<A>.)

This message was sent by Atlassian JIRA

View raw message