storm-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Abhishek Agarwal <abhishc...@gmail.com>
Subject Re: How do you add a custom class to Config?
Date Thu, 21 Apr 2016 05:38:47 GMT
In the prepare/open method, you can get the class name and initialize the
instance from class name

Excuse typos
On Apr 21, 2016 9:17 AM, "Navin Ipe" <navin.ipe@searchlighthealth.com>
wrote:

> Thanks Abhishek, but I don't quite understand what you meant. Do you mean
> I pass a class name of a static class and then use it?
>
> Also took a look at the source code and this is what it says:
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> */** * Topology configs are specified as a plain old map. This class
> provides a * convenient way to create a topology config map by providing
> setter methods for * all the configs that can be set. It also makes it
> easier to do things like add * serializations. * * This class also provides
> constants for all the configurations possible on * a Storm cluster and
> Storm topology. Each constant is paired with an annotation * that defines
> the validity criterion of the corresponding field. Default * values for
> these configs can be found in defaults.yaml. * * Note that you may put
> other configurations in any of the configs. Storm * will ignore anything it
> doesn't recognize, but your topologies are free to make * use of them by
> reading them in the prepare method of Bolts or the open method of *
> Spouts. */public class Config extends HashMap<String, Object> {*
>
> So if Storm is supposed to ignore anything it does not recognize, then
> it's strange it's throwing an error for my serializable class.
>
> On Thu, Apr 21, 2016 at 12:01 AM, Abhishek Agarwal <abhishcool@gmail.com>
> wrote:
>
>> Pass the class name in config instead of concrete object.
>>
>> Excuse typos
>> On Apr 20, 2016 9:43 PM, "Navin Ipe" <navin.ipe@searchlighthealth.com>
>> wrote:
>>
>>> Does anyone know how to do this? If not adding a custom class, then at
>>> least about how to maintain information retrieved from a database in a
>>> single class which all spouts/bolts in all the workers will be able to
>>> access?
>>>
>>> On Tue, Apr 19, 2016 at 2:52 PM, Navin Ipe <
>>> navin.ipe@searchlighthealth.com> wrote:
>>>
>>>> I have this
>>>>
>>>> Config config = new Config();
>>>> MongoDatabaseManager mongoManager = new MongoDatabaseManager();
>>>> config.put("MongoManager", mongoManager);
>>>>
>>>> and MongoDatabaseManager is an empty class:
>>>> public class MongoDatabaseManager implements Serializable {}
>>>>
>>>> But after submitting the topology, I get this error:
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> *14:45:18.320 [timer] DEBUG o.a.s.scheduler.SupervisorDetails -
>>>> Creating a new supervisor (null-5b950364-129e-4e04-b10c-d1e496c12043) with
>>>> resources: {"supervisor.cpu.capacity" 400.0,
>>>> "supervisor.memory.capacity.mb" 3072.0}14:45:18.321 [main] ERROR
>>>> o.a.s.s.o.a.z.s.NIOServerCnxnFactory - Thread Thread[main,5,main]
>>>> diedjava.lang.IllegalArgumentException: Topology conf is not
>>>> json-serializable    at
>>>> org.apache.storm.testing$submit_local_topology.invoke(testing.clj:299)
>>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>>> org.apache.storm.LocalCluster$_submitTopology.invoke(LocalCluster.clj:49)
>>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>>> org.apache.storm.LocalCluster.submitTopology(Unknown Source)
>>>> ~[storm-core-1.0.0.jar:1.0.0]    at
>>>> com.slh.Basicstorm.BasicStorm.main(BasicStorm.java:44) ~[main/:na]*
>>>>
>>>> Why can't I add a custom class to config? The idea here is to store
>>>> some info in the class so that all workers can access the information
>>>> (because the class is part of the topology) even though they are part of
>>>> different JVM's (the class can be accessed in the open() or prepare()
>>>> functions of spouts or bolts, but first, I don't understand why this crash
>>>> is happening).
>>>>
>>>> --
>>>> Regards,
>>>> Navin
>>>>
>>>
>>>
>>>
>>> --
>>> Regards,
>>> Navin
>>>
>>
>
>
> --
> Regards,
> Navin
>

Mime
View raw message