After writing my own producer with a custom round-robin partitioner, it seems the problem is solved now. Here’s my topology:
BrokerHosts brokerHosts = new ZkHosts(_kafkaZkConnect);
TridentKafkaConfig kafkaConfig = new TridentKafkaConfig(brokerHosts, _topic);
Stream inputStream = _topology.newStream("bytes", new TransactionalTridentKafkaSpout(kafkaConfig))
.each(new Fields("bytes"), new Reporter(graphiteConfig, "tuples", "received")).name("reporting-to-graphite");
Huy, Le Van
INSIGHT@ NUI Galway
Lower Dangan, IDA Business Park,Galway, Ireland
On Wednesday, Dec 10, 2014 at 6:52 p.m., P. Taylor Goetz <email@example.com>, wrote:
Can you share your topology code? It sound like you may not be setting the parallelism hint in the right place.
Could someone help me please?
On Thursday, Dec 4, 2014 at 3:35 p.m., Huy Le Van <firstname.lastname@example.org>, wrote:
However, my Kafka topic has 16 partitions and I already set parallelismHint of TransactionalTridentKafkaSpout to 16. What am I doing wrong here? Please advise.