samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naveen Somasundaram" <navg...@gmail.com>
Subject Re: Review Request 35676: Checkpoint migration
Date Thu, 02 Jul 2015 21:10:55 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/35676/
-----------------------------------------------------------

(Updated July 2, 2015, 9:10 p.m.)


Review request for samza.


Repository: samza


Description
-------

SAMZA-615: Checkpoint migration


Diffs
-----

  build.gradle a5f54106a822dc91ff82270df27217a8765a0d80 
  checkstyle/import-control.xml 3374f0c432e61ac4cda275377604cfd481f0cddf 
  samza-core/src/main/java/org/apache/samza/coordinator/stream/CoordinatorStreamMessage.java
6c1e488d00d8593d59c89b57e673e0b6b90fd7d2 
  samza-core/src/main/scala/org/apache/samza/job/JobRunner.scala 1c178a661e449c6bdfc4ce431aef9bb2d261a6c2

  samza-core/src/main/scala/org/apache/samza/migration/JobRunnerMigration.scala PRE-CREATION

  samza-core/src/main/scala/org/apache/samza/migration/MigrationPlan.java PRE-CREATION 
  samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamSystemFactory.java
647cadb3a4e51bec8204197d77ad35a6b29afcec 
  samza-core/src/test/java/org/apache/samza/coordinator/stream/MockCoordinatorStreamWrappedConsumer.java
e454593ebd2b09f0cfb7a7531f05394348253e9b 
  samza-core/src/test/scala/org/apache/samza/job/TestJobRunner.scala 52057edbbcd08b7d6edde7e898466d26534f52f6

  samza-kafka/src/main/scala/old/checkpoint/KafkaCheckpointLogKey.scala PRE-CREATION 
  samza-kafka/src/main/scala/old/checkpoint/KafkaCheckpointManager.scala PRE-CREATION 
  samza-kafka/src/main/scala/old/checkpoint/KafkaCheckpointManagerFactory.scala PRE-CREATION

  samza-kafka/src/main/scala/old/checkpoint/KafkaCheckpointMigration.scala PRE-CREATION 
  samza-kafka/src/test/scala/old/checkpoint/TestKafkaCheckpointManager.scala PRE-CREATION

  samza-test/src/main/config/join/common.properties ac87e81aeea4015b2fc83942a62e3d16c4fbbd2b


Diff: https://reviews.apache.org/r/35676/diff/


Testing (updated)
-------

Unit test added, test will hello-samza in progress

Hello-samza test:

OUTPUT FROM 0.9:
~/Documents/hello-samza(branch:latest) » deploy/kafka/bin/kafka-console-consumer.sh  --zookeeper
localhost:2181 --topic __samza_checkpoint_ver_1_for_wikipedia-parser_1 --from-beginning  
                                                                                         
                                                                 nsomasun@nsomasun-mn1

{"Partition 0":0}
{}
{"SystemStreamPartition [kafka, wikipedia-raw, 0]":{"system":"kafka","partition":"0","offset":"1674","stream":"wikipedia-raw"}}
{"SystemStreamPartition [kafka, wikipedia-raw, 0]":{"system":"kafka","partition":"0","offset":"1808","stream":"wikipedia-raw"}}
{"SystemStreamPartition [kafka, wikipedia-raw, 0]":{"system":"kafka","partition":"0","offset":"1950","stream":"wikipedia-raw"}}
{"SystemStreamPartition [kafka, wikipedia-raw, 0]":{"system":"kafka","partition":"0","offset":"2103","stream":"wikipedia-raw"}}

------------------------------------------------------------

OUTPUT FROM 0.10.0:

~/Documents/hello-samza(branch:latest) » deploy/kafka/bin/kafka-console-consumer.sh  --zookeeper
localhost:2181 --topic __samza_coordinator_wikipedia-parser_1 --from-beginning           
                                                                                         
                                                                 nsomasun@nsomasun-mn1
{"host":"172.21.136.130","source":"CHECKPOINTMIGRATION","values":{"kafka.wikipedia-raw.0":"2103"},"username":"nsomasun","timestamp":1435870948879}


Thanks,

Naveen Somasundaram


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message