flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] [flink-statefun] sjwiesman opened a new pull request #231: [FLINK-22554] Add topic subscription to KafkaIngress
Date Mon, 03 May 2021 20:11:17 GMT

sjwiesman opened a new pull request #231:
URL: https://github.com/apache/flink-statefun/pull/231


   Update the Kafka ingress to support Flinks Kafka topic discovery. This removes a common
source of downtime from the statefun cluster, configuring new topics.  Users can now configure
their ingress to use a topic pattern and the source will consume from all topics that match
the regex and auto-discover new topics every `discoveryInterval`. 
   
   ```yaml
    version: "3.0"
    
    module:
      meta:
        type: remote
    spec:
      ingresses:
      - ingress:
          meta:
            type: io.statefun.kafka/ingress
            id: com.example/users
          spec:
            address: kafka-broker:9092
            consumerGroupId: my-consumer-group
            startupPosition:
              type: earliest
            topicPattern:
               pattern: topic-*
               discoveryInterval: 30s
               valueType: com.example/User
               targets:
                 - com.example.fns/greeter
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
users@infra.apache.org



Mime
View raw message