samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jayati Tiwari <>
Subject Writing a Samza application to work as both Kafka Producer and Consumer
Date Thu, 07 Nov 2013 12:25:12 GMT

I have tried running the Hello-Samza project which worked successfully for me. Now I am targeting
a Samza application that can consume messages from a Kafka console producer and process it
to emit the results back to a Kafka console consumer.

I followed the thread "Writing a simple KafkaProducer in Samza", from where I could get an
example of the Kafka consumer in "".

On similar lines, I have been trying to create a Samza program in Java called
which is attached herewith. I have created the class in the package "samza.examples.wikipedia.task"
of the samza-wikipedia project. And I follow the same instructions to build the project as
those for Hello-Samza.

The config/properties file I use is also attached. I need to add "systems.kafka.samza.msg.serde=string"
since the messages I emit from the kafka producer console are strings, as opposed to "systems.kafka.samza.msg.serde=json"
used in the "WikipediaParserStreamTask" example.

Even after having tried out many changes to the properties file, the Kafka producer and Samza
application don't seem to connect to each other. However, I can see SamzaAppMaster and SamzaContainer
processes running on the machine.

I can get that since the Samza application is not able to receive messages from the Kafka
producer running locally, its process() method is not invoked.

Can you help me in making this work please?



NOTE: This message may contain information that is confidential, proprietary, privileged or
otherwise protected by law. The message is intended solely for the named addressee. If received
in error, please destroy and notify the sender. Any use of this email is prohibited when received
in error. Impetus does not represent, warrant and/or guarantee, that the integrity of this
communication has been maintained nor that the communication is free of errors, virus, interception
or interference.

View raw message