kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bigdata hadoop <hadoopst...@gmail.com>
Subject Offsets not matching with the count
Date Tue, 08 Dec 2015 18:50:38 GMT
Hi All

I have a topic with encrypted data.
I am using spring-xd streams kafka source and my processor which decrypts
the message and writes to file.
So the issue I am facing is the offsets or throughput isn't matching with
the topic offsets.
I also made a tap on the stream for counting the messages read from kafka
source,where the count shows something different from the source offset.
Here are the streams I deployed:

stream create --name modulename --definition "kafka
--zkconnect=xx.xx.xx:2181,xx.xx.xx:2181,xx.xx.xx:2181 --topic=topicname
--groupId=modulegroup --initialOffsets=0@12,1@18,2@72| decryptor|file"

stream deploy modulename --properties "module.*.count=3"

stream create --name modulenametap --definition "tap:stream:modulename  >
counter --name=modulenamecount" --deploy

counter display modulenamecount

Any suggestions would be helpfull.



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message