samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From GitBox <...@apache.org>
Subject [GitHub] cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor for the KafkaSystemConsumer.
Date Wed, 20 Feb 2019 01:03:47 GMT
cameronlee314 commented on a change in pull request #918: SAMZA-2094: Implement the StartpointVisitor
for the KafkaSystemConsumer.
URL: https://github.com/apache/samza/pull/918#discussion_r258283701
 
 

 ##########
 File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemConsumer.java
 ##########
 @@ -159,30 +179,16 @@ private void startSubscription() {
    */
   void startConsumer() {
     // set the offset for each TopicPartition
-    if (topicPartitionsToOffset.size() <= 0) {
+    if (topicPartitionToStartpointMap.size() <= 0) {
       LOG.error ("{}: Consumer is not subscribed to any SSPs", this);
     }
 
-    topicPartitionsToOffset.forEach((tp, startingOffsetString) -> {
-      long startingOffset = Long.valueOf(startingOffsetString);
-
-      try {
-        synchronized (kafkaConsumer) {
-          kafkaConsumer.seek(tp, startingOffset); // this value should already be the 'upcoming'
value
-        }
-      } catch (Exception e) {
-        // all recoverable execptions are handled by the client.
-        // if we get here there is nothing left to do but bail out.
-        String msg =
-            String.format("%s: Got Exception while seeking to %s for partition %s", this,
startingOffsetString, tp);
-        LOG.error(msg, e);
-        throw new SamzaException(msg, e);
-      }
-
-      LOG.info("{}: Changing consumer's starting offset for tp = {} to {}", this, tp, startingOffsetString);
-
+    topicPartitionToStartpointMap.forEach((topicPartition, startpoint) -> {
+      Partition partition = new Partition(topicPartition.partition());
+      SystemStreamPartition systemStreamPartition = new SystemStreamPartition(systemName,
topicPartition.topic(), partition);
+      startpoint.apply(systemStreamPartition, kafkaStartpointRegistrationHandler);
       // add the partition to the proxy
-      proxy.addTopicPartition(topicPartitionsToSSP.get(tp), startingOffset);
+      proxy.addTopicPartition(systemStreamPartition, kafkaConsumer.position(topicPartition));
 
 Review comment:
   Looks like the previous comment about this got marked as "outdated".
   Can this be moved into the registration handler? Seems like this functionality would fall
into the responsibility of that object.

----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
users@infra.apache.org


With regards,
Apache Git Services

Mime
View raw message