storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Navin Ipe <navin....@searchlighthealth.com>
Subject Re: How do you add a custom class to Config?
Date Thu, 21 Apr 2016 03:47:18 GMT
Thanks Abhishek, but I don't quite understand what you meant. Do you mean I
pass a class name of a static class and then use it?

Also took a look at the source code and this is what it says:
















*/** * Topology configs are specified as a plain old map. This class
provides a * convenient way to create a topology config map by providing
setter methods for * all the configs that can be set. It also makes it
easier to do things like add * serializations. * * This class also provides
constants for all the configurations possible on * a Storm cluster and
Storm topology. Each constant is paired with an annotation * that defines
the validity criterion of the corresponding field. Default * values for
these configs can be found in defaults.yaml. * * Note that you may put
other configurations in any of the configs. Storm * will ignore anything it
doesn't recognize, but your topologies are free to make * use of them by
reading them in the prepare method of Bolts or the open method of *
Spouts. */public class Config extends HashMap<String, Object> {*

So if Storm is supposed to ignore anything it does not recognize, then it's
strange it's throwing an error for my serializable class.

On Thu, Apr 21, 2016 at 12:01 AM, Abhishek Agarwal <abhishcool@gmail.com>
wrote:

> Pass the class name in config instead of concrete object.
>
> Excuse typos
> On Apr 20, 2016 9:43 PM, "Navin Ipe" <navin.ipe@searchlighthealth.com>
> wrote:
>
>> Does anyone know how to do this? If not adding a custom class, then at
>> least about how to maintain information retrieved from a database in a
>> single class which all spouts/bolts in all the workers will be able to
>> access?
>>
>> On Tue, Apr 19, 2016 at 2:52 PM, Navin Ipe <
>> navin.ipe@searchlighthealth.com> wrote:
>>
>>> I have this
>>>
>>> Config config = new Config();
>>> MongoDatabaseManager mongoManager = new MongoDatabaseManager();
>>> config.put("MongoManager", mongoManager);
>>>
>>> and MongoDatabaseManager is an empty class:
>>> public class MongoDatabaseManager implements Serializable {}
>>>
>>> But after submitting the topology, I get this error:
>>>
>>>
>>>
>>>
>>>
>>>
>>> *14:45:18.320 [timer] DEBUG o.a.s.scheduler.SupervisorDetails - Creating
>>> a new supervisor (null-5b950364-129e-4e04-b10c-d1e496c12043) with
>>> resources: {"supervisor.cpu.capacity" 400.0,
>>> "supervisor.memory.capacity.mb" 3072.0}14:45:18.321 [main] ERROR
>>> o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Thread Thread[main,5,main]
>>> diedjava.lang.IllegalArgumentException: Topology conf is not
>>> json-serializable    at
>>> org.apache.storm.testing$submit_local_topology.invoke(testing.clj:299)
>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>> org.apache.storm.LocalCluster$_submitTopology.invoke(LocalCluster.clj:49)
>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>> org.apache.storm.LocalCluster.submitTopology(Unknown Source)
>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>> com.slh.Basicstorm.BasicStorm.main(BasicStorm.java:44) ~[main/:na]*
>>>
>>> Why can't I add a custom class to config? The idea here is to store some
>>> info in the class so that all workers can access the information (because
>>> the class is part of the topology) even though they are part of different
>>> JVM's (the class can be accessed in the open() or prepare() functions of
>>> spouts or bolts, but first, I don't understand why this crash is happening).
>>>
>>> --
>>> Regards,
>>> Navin
>>>
>>
>>
>>
>> --
>> Regards,
>> Navin
>>
>


-- 
Regards,
Navin

Mime
View raw message