flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2837) FlinkTopologyBuilder cannot handle multiple input streams
Date Tue, 24 Nov 2015 12:35:11 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15024420#comment-15024420
] 

ASF GitHub Bot commented on FLINK-2837:
---------------------------------------

Github user mxm commented on a diff in the pull request:

    https://github.com/apache/flink/pull/1398#discussion_r45729000
  
    --- Diff: flink-contrib/flink-storm-examples/src/main/java/org/apache/flink/storm/util/FiniteFileSpout.java
---
    @@ -32,46 +23,17 @@
     public class FiniteFileSpout extends FileSpout implements FiniteSpout {
     	private static final long serialVersionUID = -1472978008607215864L;
     
    -	private String line;
    -	private boolean newLineRead;
    -
     	public FiniteFileSpout() {}
     
     	public FiniteFileSpout(String path) {
     		super(path);
     	}
     
    -	@SuppressWarnings("rawtypes")
    -	@Override
    -	public void open(final Map conf, final TopologyContext context, final SpoutOutputCollector
collector) {
    -		super.open(conf, context, collector);
    -		newLineRead = false;
    -	}
    -
    -	@Override
    -	public void nextTuple() {
    -		this.collector.emit(new Values(line));
    -		newLineRead = false;
    -	}
    -
     	/**
     	 * Can be called before nextTuple() any times including 0.
     	 */
     	@Override
     	public boolean reachedEnd() {
    -		try {
    -			readLine();
    -		} catch (IOException e) {
    -			throw new RuntimeException("Exception occured while reading file " + path);
    -		}
    -		return line == null;
    +		return finished;
     	}
    --- End diff --
    
    I don't understand why I should copy code instead. It is more elegant this way. 


> FlinkTopologyBuilder cannot handle multiple input streams
> ---------------------------------------------------------
>
>                 Key: FLINK-2837
>                 URL: https://issues.apache.org/jira/browse/FLINK-2837
>             Project: Flink
>          Issue Type: Bug
>          Components: Storm Compatibility
>            Reporter: Matthias J. Sax
>            Assignee: Maximilian Michels
>
> FlinkTopologyBuilder cannot handle multiple input streams correctly. Instead of union
the incoming streams, it replicates the consuming bolt and each (logical) instance processes
one of the input streams.
> For example:
> {noformat}
> final FlinkTopologyBuilder builder = new FlinkTopologyBuilder();
> builder.setSpout(spoutId1, new FiniteRandomSpout(0, 10));
> builder.setSpout(spoutId2, new FiniteRandomSpout(1, 8));
> builder.setSpout(spoutId3, new FiniteRandomSpout(2, 13));
> builder.setBolt(boltId, new MergerBolt())
> 	.shuffleGrouping(spoutId1)
> 	.shuffleGrouping(spoutId2)
> 	.shuffleGrouping(spoutId3);
> builder.setBolt("sink", new BoltPrintSink(new SimpleOutputFormatter()))
> 	.shuffleGrouping(boltId);
> {noformat}
> will only print the data from a single source instead of all sources.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message