flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pnowojski <...@git.apache.org>
Subject [GitHub] flink issue #5316: [FLINK-8022][kafka-tests] Disable at-least-once tests for...
Date Tue, 23 Jan 2018 13:45:21 GMT
Github user pnowojski commented on the issue:

    https://github.com/apache/flink/pull/5316
  
    What do you mean? Do you want to re-run this test three times and fail it if only two
or three runs have failed? 
    
    This test fails on reading the data back from Kafka during the checking assertion for
`at-least-once`. The loop looks like this:
    
    ```
    		// until we timeout...
    		while (System.currentTimeMillis() < startMillis + timeoutMillis) {
    			// query kafka for new records ...
    			Collection<ConsumerRecord<Integer, Integer>> records = kafkaServer.getAllRecordsFromTopic(properties,
topic, partition, 100);
    
    			for (ConsumerRecord<Integer, Integer> record : records) {
    				actualElements.add(record.value());
    			}
    
    			// succeed if we got all expectedElements
    			if (actualElements.containsAll(expectedElements)) {
    				return;
    			}
    		}
    ```
    
    with `timeoutMillis` set to 60 seconds and `100` parameter of `getAllRecordsFromTopic`
method means "wait for 100ms for records". When this fails it means something has broken and
we are not able to read the expected data for 60 seconds, either because of failure:
    
    - on Kafka side
    - on our test code side (the way how we setup network proxies for triggering network failures)
    - on our production code



---

Mime
View raw message