samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Garry Turkington <g.turking...@improvedigital.com>
Subject Problem testing creation of ConsumerConfig object (re SAMZA-77)
Date Tue, 19 Nov 2013 00:53:44 GMT
Hi,

I thought SAMZA-77 looked like fun so made the changes required to add the stringified UUID
to the client and group IDs.  All nice and easy.  Then I tried adding some test cases and
think I'm managing to make this more difficult than it needs to be. :)

In my test case I have the following, which I believe is the right chain of calls to create
a KafkaConfig object (I created a simple config file in the resources directory):

  val factory = new PropertiesConfigFactory()
    val config = factory.getConfig(URI.create("file://%s/src/test/resources/test.properties<file:///\\%25s\src\test\resources\test.properties>"
format new File
(".").getCanonicalPath))
val kafkaConfig = new KafkaConfig(config)

All well and good but then I try and call the appropriate method on the KafkaConfig object
(with some uncertainty of what the argument needs map to):

val config1 = kafkaConfig.getKafkaSystemConsumerConfig("appname")

This causes an exception to be thrown, the highlights being:

java.lang.IllegalArgumentException: requirement failed: Missing required property 'zookeeper.connect'
        at scala.Predef$.require(Predef.scala:145)
        at kafka.utils.VerifiableProperties.getString(VerifiableProperties.scala:175)
        at kafka.utils.ZKConfig.<init>(ZkUtils.scala:775)
        at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:73)
        at kafka.consumer.ConsumerConfig.<init>(ConsumerConfig.scala:77)
        at org.apache.samza.config.KafkaConfig.getKafkaSystemConsumerConfig(KafkaConfig.scala:91)

This suggests that to create a ConsumerConfig requires the whole ZK/Kafka stack to be instantiated
- is this the case and if so pointers to the minimum config set required to get it working?
 I tried to piece something together from the other test cases but couldn't find the right
magic.

Or is there an easier way to do all this? :)

Thanks
Garry

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message