kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From siddharth ubale <siddharth.ub...@gmail.com>
Subject Kafka-Storm Integration
Date Tue, 20 Jan 2015 14:25:50 GMT
Hi,

I am runnning a Kafka-storm topology .

The topology works exactly as expected in LocalCluster mode. However in
Distributed mode the Workers are not starting , supervisor has not started.
Only nimbus starts

I am trying to run the topology via eclipse.

//Providing the details for zkHosts
BrokerHosts hosts = new ZkHosts("hbase:2181","/brokers");
 SpoutConfig spoutconf = new SpoutConfig(hosts,TOPIC_NAME,"/test101","sid");
spoutconf.scheme = new SchemeAsMultiScheme(new StringScheme());
spoutconf.forceFromStart = true;
KafkaSpout Kspout = new KafkaSpout(spoutconf);
 builder.setSpout("spout", Kspout, 1);

PhoenixBolt phoenixBolt = new PhoenixBolt("Phoenix-Bolt", true,true,new
Fields("json"));
builder.setBolt("Phoenix-Bolt", phoenixBolt, 1).shuffleGrouping("spout");

Config config = new Config();

config.put(Config.NIMBUS_THRIFT_PORT,6627);
config.put(Config.STORM_ZOOKEEPER_PORT,2181);
config.put(Config.STORM_ZOOKEEPER_SERVERS,Arrays.asList(new
String[]{"hbase"}));
config.put(Config.NIMBUS_HOST, "dwh3");
config.put("storm.zookeeper.root","/storm1");
config.put(Config.STORM_ZOOKEEPER_ROOT, "/storm1");

System.setProperty("storm.jar","/usr/local/apache-storm-0.9.3/bin/storm");
config.setDebug(true);
config.setNumWorkers(5);
config.put(Config.TOPOLOGY_DEBUG, true);
System.out.println(config.toString());

//cluster.submitTopology("kafka-storm-Phoenix", config,
//builder.createTopology());
 StormSubmitter.submitTopology("kafka-storm-Phoenix-ClusterD", config,
builder.createTopology());


worker logs are saying " Netty reconnect for hostname:6703" is issued again
and again.


Thanks,
Siddharth

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message