flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chesnay Schepler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2899) The groupReduceOn* methods which take types as a parameter fail with TypeErasure
Date Thu, 29 Oct 2015 13:47:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14980448#comment-14980448

Chesnay Schepler commented on FLINK-2899:

Looking at the Graph class, when groupReduceOnEdges (..., TypeInformation<T> typeInfo)
is used, two separate type hints are given to the system. One via .returns(typeInfo), and
one via the ResultTypeQueryable interface in the ApplyNeighborCoGroupFunction class. When
mixing these two, the interface takes priority as far as i know, so the .returns() call is
always ignored. 

Actually, the usage of the interface doesn't make a lot of sense to me, because it only calls
TypeExtractor.createTypeInfo(...) to extract the generic typeinfo from a function, which is
the same thing that would happen when the interface wouldn't be implemented. I don't think
the function within the ApplyNeightborCoGroupFunction holds more information.

Without trying things out myself, I assume that 2 solutions exist: either always supply the
ReturnType with .returns() and scrap the ResultTypeQueryable, or ditch .returns() and pass
the TypeInfo to the function and always use the interface just like you did in your hack.

> The groupReduceOn* methods which take types as a parameter fail with TypeErasure
> --------------------------------------------------------------------------------
>                 Key: FLINK-2899
>                 URL: https://issues.apache.org/jira/browse/FLINK-2899
>             Project: Flink
>          Issue Type: Bug
>          Components: Gelly
>    Affects Versions: 0.10
>            Reporter: Andra Lungu
> I tried calling  groupReduceOnEdges (EdgesFunctionWithVertexValue<K, VV, EV, T>
edgesFunction, EdgeDirection direction, TypeInformation<T> typeInfo) in order to make
the vertex-centric version of the Triangle Count library method applicable to any kind of
key and I got a TypeErasure Exception. 
> After doing a bit of debugging (see the hack in https://github.com/andralungu/flink/tree/trianglecount-vertexcentric),
I saw that actually the call to TypeExtractor.createTypeInfo(NeighborsFunctionWithVertexValue.class,
.... in ApplyNeighborCoGroupFunction does not work properly, i.e. it returns null. From what
I see, the coGroup in groupReduceOnNeighbors tries to infer a type before "returns" is called.

> I may be missing something, but that particular feature (groupReduceOn with types) is
not documented or tested so we would also need some tests for that. 

This message was sent by Atlassian JIRA

View raw message